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1.  A  description  of  the  estimation  process  and  of  related  problems  includ¬ 
ing  the  outline  of  a  conceptual  representation  of  the  process  against  which 
specific  methods  may  be  evaluated. 

2.  A  description  of  analytical  aids  useful  in  the  intention  estimation 
process  including  judgmental  methods  as  well  as  analytic?1  aids  using 
techniques  of  extrapolation,  structural,  and  causal  analysis. 

3.  A  comparison  of  analytical  aids  in  the  context  of  the  conceptual 
estimation  process. 

4.  The  evaluation  of  specific  analytical  aids. 

5.  Development  of  a  catalogue  of  18  analytical  aids  matched  by 
methodology  to  steps  in  the  conceptual  estimation  process. 

/> 


Unclassified 


UNCLASSI FIED  _ 


SECURITY  CLASSIFICATION  of  TN  S  PAGE  Omto  En, #»•<»; 


READ  INSTRUCTIONS 
BEFORE  COMPLETING  FORM 


REPORT  DOCUMENTATION  PAGE 


.  »  OAT  NUMtCR  |2.  GOVT  ACCESSION  HQa  3.  ACClAtC:  T*3  CATALOG  NUMBCA 


GOVT  ACCESSION  NOJ  3. 

M // r> 


4.  TITUS  f«W  Subtltlo) 

METHODS  OF  ESTIMATING  STRATEGIC  INTENTIONS 


5.  TYPE  OF  REPORT  *  PERIOO  COVERED 


Technical  Report 


S.  PERFORMING  OFO.  FCFOFT  NUMBER 


7.  AUtMOWai 


Frank  J.  Stech 
Kenneth  C.  Hoffman 


S.  CONTRACT  OF  GRANT  NUMBER/*,) 


NGGG14-81-C-G508 


S.  PERFORMING  ORGANIZATION  NAME  ANO  AOORESS 

MATHTECH,  Inc. 

4630  Montgomery  Avenue 
Bethesda,  Maryland  20014 


M.  CONTROLLING  OFFICE  NAME  ANO  AOORESS 

Office  of  Naval  Research,  code  4308 
800  North  Quincy  Street 
Arlington,  Virginia  22217 


IO.  PROGRAM  ELEMENT,  PROJECT.  TASK 

AREA  *  WORK  UNIT  NUMBERS 


12.  REFORT  OATt 


May  1982 


IS.  NUMBER  OF  PAGES 

143 


MONITORING  AGENCY  NAME  *  AOORESVI/  dlltormt  tram  Controlling  Ottle •)  '5.  SECURITY  CLASS,  (at  Otto  roport) 

Unclassified 

IS*.  OECLASSI  FI  CATION  /  OOWN  GRADING 
SCHEDULE 


t«.  DISTRIBUTION  STATEMENT  (ol  Mo  Rap on) 


Uni Imlted 


Approv«}  for  public  iml« 

DUtrtbutton  UnUmltad 


17.  DISTRIBUTION  STATEMENT  (ol  tho  oboltoct  omtotod  In  Mot*  20.  II  dJ  t  tor  on  t  Irom  Moport) 


IB.  K  EY  WQROS  (Conllnoo  on  toooroo  alp*  II  mooooo 

Capabilities  &  Intentions 
Cognitive  Processes 
Intelligence  Estimates 
Intention  Estimation 


md  Ibontllr  by  blot*  hum  tor) 


Methods  of  Analysis 
Military  Intelligence 
Military  Psychology 
Naval  Intelligence 


SO.  ABSTRACT  C  Conti  moo  om  too 


otOo  It  miniMT  on4  Itontlly  by  blot *  mm* or) 


’ — 3  The  estimation  of  strategic  intentions  is  an  important  and  evolving  part  of 
the  Intelligence  process.  The  research  program  described  in  this  report  was 
undertaken  to  catalogue  a  number  of  quantitative  and  behavioral  methods  of 
analysis  used  in  support  of  various  phases  of  the  estimation  process.  Thus,  this 
research  builds  on  earlier  work  performed  by  MATHTECH  in  which  the  intention 
estimation  process  wcs  defined  and  various  supporting  methods  identified. 

Specific  tasks  undertaken  in  this  research  project  are:  _  .. 
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SUMMARY 


The  estimation  of  strategic  Intentions  Is  an  Important  and  evolving  part  of 
the  Intelligence  process.  The  motivation  for  focusing  on  estimates  of  peacetime 
Soviet  naval  Intentions  In  previous  research  studies  performed  by  MATHTECH 
arises  from  the  Increased  contacts  between  U.S.  or  Western  fleets  and  the  Soviet 
Navy.  These  more  frequent  and  extensive  contacts  In  off-shore  defensive  zones, 
sea  lanes,  and  In  proximity  to  Third  World  countries  may  occur  In  crisis  situa¬ 
tions  or  In  situations  with  the  potential  for  escalation  into  a  major  crisis 
because  of  the  behavior  and  responses  among  the  parties  Involved.  The  way  In 
which  the  navies  of  the  West  and  the  Soviet  Union  conduct  themselves  In  these 
peacetime  and  crisis  encounters  has  a  considerable  Impact  on  subsequent  escala¬ 
tion  and  responses.  The  previous  research  performed  by  MATHTECH  dealt  with  an 
assessment  of  methods  for  estimating  peacetime  Soviet  naval  Intentions.  In  the 
course  of  that  research  a  number  of  quantitative  and  behavioral  methods  were 
I  dent  I  fled  (Stech,  1981).  A  foi  low-on  research  program,  the  resu I ts  of  which 
are  described  In  this  report,  was  then  undertaken  to  catalogue  those  methods  in 
relation  to  estimation  problems  and  issues.  In  addition  criteria  were  estab¬ 
lished  to  guide  the  selection  of  appropriate  methods  to  deal  with  specific 
estimation  issues.  As  background  for  this  research,  a  quite  detailed  descrip¬ 
tion  of  the  Intention  estimation  process  was  developed. 

The  individual  tasks  that  were  performed  In  this  research  project  are: 

TASK  1 .  Description  of  the  Estimation  Process  and  of  Problems 
that  Arise  In  the  Process; 

The  conceputal  model  developed  In  this  task  to  provide  the  context  for 
Intention  estimates  Includes  3  sequential  steps; 
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o  precep+lons  (reactions  to  the  environment  and  situation  based  on 
capabilities  and  behavior), 

o  estimates  and  assessments  of  current  and  future  capabl I [ties, 
risks,  actions  and  reactions, 

o  Integration  and  judgements  related  to  evaluation  of  alternate 
hypotheses, 

o  definition  of  Intentions  based  on  logic  embodied  In  previous 
steps, 

o  formulation  of  a  strategic  concept  outlining  courses  of  action, 
o  strategy  statement  to  guide  future  action, 
o  development  of  plan  to  guide  actions,  and 
o  execution  of  the  plan. 


TASK  2:  Oescr lotion  of  Analytical  Aids: 

Analytical  aids  useful  In  the  Intention  estimation  process  vary 
considerably  In  scope,  methodology,  and  logic.  Consistent  with  the  complexity 
and  range  of  possibilities  of  Intentions,  these  aids  are  similarly  complex  and 
wide  ranging;  there  are  no  Individual  methods  that  are  to  be  preferred  even  In  a 
given  class  of  analytical  problems.  The  methods  vary  considerably  In  their 
dependence  on  data.  Judgements,  and  relational  or  behavioral  descriptions,  as 
well  as  In  their  dependence  on  quantitative  techniques.  They  range  from  purely 
subjective  or  Judgemental  methods  to  highly  quantitative  probabilistic  methods. 
A  general  categorization  of  analytical  aids  Is  as  follows: 


o  Judgemental  Methods;  supported  by  attributes  of  sagacity, 
control,  and  acumen. 

o  Analytical  Aids;  quantitative  methods  that  support  specific 
elements  of  the  estimation  process, 
o  Extrapolation  Models;  based  on  past  events  and  data,  and 
o  Structural  models;  taking  Into  account  structural  realities  and 
changes  In  leadership,  physical  capabilities,  and  other 
Influencing  factors. 


The  third  and  fourth  categories  make  extensive  use  of  quantitative  methods.  In 
one  case  Involving  trend  extrapolation.  In  the  other  attempting  to  represent  the 
physical  and  behavioral  relationships  that  affect  Intentions  In  a  given  time  and 
place.  Clearly,  these  categories  are  not  mutually  exclusive  and  Intention 
estimates  will  be  based  on  some  combination  of  methods  drawn  from  these  cate¬ 
gories.  The  art  of  estimation  Is  to  select  and  blend  appropriate  methods  and 
the  objective  of  this  research  project  Is  to  assist  In  that  process  through  the 
cataloguing  and  organization  of  analytical  aids  used  to  support  and  extend  human 
Judgement  and  Interpretation. 

The  specific  analytical  aids  covered  In  this  review  are  listed  as  follows 
In  relation  to  the  elements  of  Intent  estimating: 


1.  Perceiving  Oata:  statistical  sampling,  record  of  events,  coding 
categories. 
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2.  Weighting  Data:  poi Icy  capturing,  Bayesian  methods,  correlation 
and  variance  analysis. 

3.  Characterising  Data:  memory  aids,  fuzzy  sets,  factor  analysis. 

4.  Assessing  Covariations:  actuarial  models,  backcastlng, 
bootstrapping. 

5.  Cause  and  Effect  Assessment:  causa'  search,  causal  analysis, 
search  trees,  stepping  analysis,  hypothesis,  regression  analysis. 

6.  Predictions:  Backcastfng,  boot  strapping,  decision  analysis. 

7.  Theories:  scenarios.  Judgement  heuristics,  etc. 

TASK  3:  Comparison  of  Aids  and  Problems: 

This  task  compares  the  analytical  aids  In  the  context  of  the  Intention 
estimation  process.  General  criteria  that  apply  to  the  comparison  Include: 


o  scope  and  content  of  method  In  relation  to  estimation  problem, 
(can  method  Incorporate  Information  on  capabilities,  force  struc¬ 
ture,  deployment,  logistics,  manpower,  risk  propensity,  strategic 
patterns,  responsive  patterns,  etc), 
o  descriptive  power 

o  Inferential  power 

o  treatment  of  constraints  (operational,  tactical,  and  strategic), 
o  logic  structure  (deterministic,  probabilistic.  Inference,  etc), 

o  transparency  of  rel attonsh Ips  and  assumptions, 

o  analytical  method  (state  of  development  and  prior  experience), 
o  appl I  cat  Ion  and  documentation  of  method, 
o  validation,  verlf l cat  I on,  and  peer  review  of  method, 
o  cost  of  Initial  system  and  upkeep, 
o  treatment  of  risk  and  uncertainty,  and 

o  sensitivity  (robustness  of  result  and  effect  of  changes  In 
assumpt 1 ons ) . 


TASK  4:  Evaluation  of  Aids: 

Problems  with  particular  analytical  aids  and  an  assessment  of  strength  and 
weaknesses  are  discussed  In  the  context  of  the  Intention  estimation  process  with 
reference  to  historical  experiences. 


TASK  5:  Catalogue  of  Aids  and  Process  Summary i 

A  summary  table  relating  specific  analytical  aids,  analytical  problems,  and 
criteria  was  developed  and  Is  Included  In  this  report. 
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Chapter  II  of  this  report  covers  Tasks  1  and  2.  the  description  of  the 
Intention  estimation  process  and  the  description  of  analytical  aids  pertinent  to 
that  process.  Chapter  III  covers  Tasks  3  and  portions  of  4,  the  comparison  of 
analytical  aids  and  their  evaluations.  The  summary  catalogue  of  aids  Is  pre¬ 
sented  In  Chapter  IV. 

Major  conclusions  that  are  drawn  from  this  project  are: 


1.  Estimation  of  Intentions  may  usefully  draw  upon  a  variety  of 
analytical  aids. 

2.  The  selection  of  specific  analytical  aids  and  their  application 
to  deal  with  estimation  problems  depends  upon  such  factors  as 
scope  of  problem,  data  aval labl I  tty,  and  timing.  This  process 
cannot  be  described  In  cook-book  fashion  with  any  absolute  guar¬ 
antees  of  success.  While  the  process  depends  a  great  deal  on 
creativity,  criteria  have  been  developed  and  can  prove  quite 
useful  as  a  checklist  for  the  selection  and  use  of  aids. 

3.  The  criteria  and  evaluation  methods  employed  In  this  project  to 
catalogue  and  evaluate  aids  In  the  estimation  of  peacetime  Soviet 
naval  Intentions  are  sufficiently  general  to  apply  to  a  wide 
range  of  Intelligence,  logistic,  and  strategic  analytical  aids  of 
Interest  to  the  office  of  Naval  Research. 


II.  DESCRIPTION  OF  THE  INTENTION  ESTIMATION  PROCESS  ANO  ANALYTIC  AIDS 


Th«  estimation  of  Intentions  by  Intelligence  analysts  can  be  characterized 
In  a  variety  of  ways  and  In  terms  of:  the  estimation  steps  taken  by  the 
analysts,  the  analysts'  logic  and  reasoning  processes,  and  the  types  of  evidence 
and  Information  analysts  might  consider  to  form  an  Intel  I  Igence  estimate  of 
intentions.  This  chapter  uses  these  three  outlooks  on  Intention  estimation  to 
describe  the  estimation  process  and  to  Identify  problems  that  might  affect 
various  steps  of  the  process. 

A.  kJOEL  OF  INTENTIONS 

This  model  Is  Intended  to  provide  the  context  or  background  for  the 
comparison  and  evaluation  of  analytical  aids.  Intentions  can  be  given  very 
complex  definitions,  or  simple  ones?  e.g.,  Lawrence  (1972:  83)  writes  that 
Intentions  are  "desirous  foreknowledge  or  expectant  desire."  Basically 
Intentions  and  psychological  states  that  Involve 

Mental  images  of  future  events  In  which  the  Intender 
pictures  himself  as  a  participant  and  makes  choices  as  to 
which  image  he  will  try  to  bring  to  reality.  (Griffin  1976: 

5)* 

An  Intelligence  professional  (Gazlt,  1980)  has  divided  Intentions  estimation 
txtQ  Identifying  a  decision  already  taken,  analyzing  responses  taken  In  reaction 
to  actions  by  others,  and  analyzing  the  outcome  of  a  developing,  ongoing 
situation.  9shavl oral  I y,  I ntentlonai l ty  manifests  several  features: 
expectations  of  the  outcome  of  an  act,  selection  among  alternatives  for  the 
fulfillment  of  a  goal,  sustained  effort  In  a  given  direction  In  the  sequence  of 
actions  taken  to  make  the  resulting  state  resemble  the  expected  state,  and  c 
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flexible  plan  for  using  alternative  means  and  actions  to  compensate  and  correct 
for  discrepancies  between  expectations  and  results  (Foster  and  Brandes,  1980: 
326).  Longer,  philosophical  d I ssertat ions  on  the  meaning  of  "Intentions" 
(Anscombe,  1969;  Lawrence,  1972;  Stech,  1979,  ch.  2)  are  interesting  but  not 
essential  to  outlining  a  descriptive  model  of  the  elements  of  Intentions  which 
an  intelligence  analyst  may  need  to  analyze. 

In  the  following  model  we  are  assuming  that  the  adversary,  or  enemy,  or 
ally,  that  is,  the  party  whose  intentions  the  intelligence  analyst  hopes  to 
ascertain,  is  not  psychopathologlcal.  Model  of  intentions  can  be  created  for 
analyzing  the  psychopathol og Ic  adversary  (e.g.,  Langer's,  1972,  model  of  the 
"mind  of  Adolf  Hitler"  created  dur i ng  Wcr I d  War  II  to  a  I  d  A I  I  led  f  ntel  1 1  gencn 
estimations),  but  each  such  model  is  a  special  case. 

Our  objective  is  to  outline  a  model  that  can  hope  to  deal  with  the 
estimation  of  the  intentions  of  most  normal  actors,  functioning  without  obvious 
psychotic  d I sturbances.  We  do  Jifli  assume  that  the  adversary  is  wholly  rational, 
i  n  the  logical,  mathemut  leal,  or  econoa  1  c  mean  I  ng  of  "rat  I  ona  1  !  ty ."  In  fact, 
one  I  Imitation  of  the  "rational  actor  model"  (as  used,  e.g.,  by  At  r  Ison,  1971, 
to  assess  the  Cuban  Missile  Crisis;  Is  that  nfl  human  actors  are  strictly 
rational  axce&t  when  they  do  mathematics,  symbolic  logic,  or  economic  utility 
calculus,  and  even  risen  biases,  errors,  and  paradoxes  can  lead  to  behavior  that 
Is  not  completely  rational.  The  only  real  requirement  for  the  model  Is  that  the 
actor,  whose  behavior  the  analyst  Is  attempting  to  predict  through  the  analysis 
of  tntentions,  not  be  widely  perceived  as  totally  Irrational,  l.e.,  the  model 
does  not  work  If  the  adversary  Is  crazy. 

The  qualification  in  the  preceding  paragraph  is  fairly  Important  from  the 
analysts*  standpoint.  The  customers  of  intelligence  estimates,  particularly 
those  which  hope  to  estimate  Intentions,  will  often  Justify  their  ignoring  such 
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estimates  with  the  assertion  that  Prime  Minister  So-and-so  Is  "unpredictable" 
or.  In  the  extreme  case.  President  Such-and-such  Is  simply  "crazy".  Neither 
argument  is  tenable;  the  first  asserts  that  we  are  as  wise  as  we  will  ever  be, 
and  that  the  fact  < 1  f  it  Is  a  fact)  that  no  accurate  predictions  were  made  In 
+hn  .part  means  none  will  ever  be  made  In  the  future.  The  second  ignores  the 
fact  ♦hat  truly  cr^zy  people  are  quite  easy  to  predtct,  use  very  circumscr I  bed 
routines  of  behavior,  ana  suffer  from  the  lack  of  behavioral  flexibility  and 
choice,  i.e.,  their  behav 'or  is  compel  led,  compulsive,  and  henci  predictable. 
The  arguments  may  justify  scepticism  in  the  predictions,  but  not  complere 
Ignorance  of  them. 

This  model  (Figure  1)  has  two  main  sources.  Its  general  characteristics 
come  from  a  model  used  I n  Wori  d  War  1 1  to  predict  Axis  Intentions  through  The 
analysis  of  propaganda,  and  it  has  been  subsequently  systematized  by  George 
(1959).  Grafted  onto  this  basic  skeleton  are  concepts  taken  from  the  general 
areas  of  decision-making  and  problem-sol vlng  behavior.  That  Is,  the  adversary 
Is  perceived  to  be  solving  a  problem:  what  actions  are  needed  to  reach  some 
goal;  and  making  a  decision:  what  action  should  be  implemented  to  reach  the 
goal.  The  analyst**  task  is  to  try  to  penetrate  the  adversary's  prob I em-so I v I ng 
and  decision-making  processes  and  procedures,  I.e.,  to  recreate  the  steps  the 
adversary  has  taken  to  solve  the  problems  and  make  a  decision. 

The  top  tine  of  the  figure  shows  Intentions,  which  can  only  be  inferred,  as 
intervening  between  the  real  world  situation  and  the  adversary's  actions,  both 
of  which  can  be  observed  directly,  as  well  as  inferred  from  other  evidence.  The 
dotted  arrow  reflects  the  fact  that  some  of  the  adversary's  actions  are  not 
intentional  but  instead  are  accidental  or  miscarriages  of  intended  actions. 
Unintended  action  is  difficult  to  separate  analytically  from  Intanded  action; 
there  Is  a  tendency  to  attribute  all  action  to  sco#  motive  or  Intention  of  the 
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Figure  i.  Conceptual  Model  of  Intentions 
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ac+or  (see  Stech,  1980:  104-120  for  a  discussion  of  this  tendency  In  naval 
analysis). 

The  second  line  of  the  figure  reflects  the  approach  taken  here  to 
Intentions,  that  Intentional  behavior  ref  I  ects  problem  formulation,  decision¬ 
making,  and  then  planning  for  the  execution  of  the  decision  taken.  These  steps 
are  conditioned  by  the  adversary's  perceptions  of  the  real  world  situation 
(I.e.,  the  adversary  considers  reality  In  formulating  the  problem)  and  are 
presumed  to  Influence  the  actions  taken. 

The  sequential  steps  of  problem  formulation  (logic  cf  estimation, 
estimation,  and  declslon/pl ann Ing  have  been  the  subjects  of  a  variety  of 
theories  and  concepts  used  by  political  scientists  to  assess  governmental 

actions.  Some  of  these  political  science  formulations  are  reflected  on  the 

% 

third  line  of  the  figure.  For  example,  generalizations  "bout  the  typical 
perceptions  and  estimations  made  by  a  nation  faced  with  various  situations 
contribute  to  understanding  how  the  nation  formulates  problems,  and  offers 
Information  on  the  Important  dimensions  and  value  criteria  the  nation  will  tend 
to  use  to  structure  Its  decisions.  Similarly  the  operational  code  of  a  national 
elite  provides  Information  on  the  instrumental  I  ties  and  basic  strategic  values 
of  the  top  decision-makers.  Again,  such  information  contributes  to 
understanding  how  a  real  world  problem  might  be  solved,  or  more  basically,  what 
will  be  perceived  as  a  problem  In  the  first  place. 

At  the  bottom  of  the  figure  are  seven  steps  which  reflect  the  process  that 
Intervenes  between  the  sttmulus  of  the  real  world  situation  and  the  action  taken 
In  response.  These  are  represented  in  Figure  2  as  sequential  steps  Indicating 
activities  preparatory  to  and  subsequent  to  the  definition  of  Intentions.  An 
eighth  step,  action,  has  been  added  to  Figure  2  for  completeness.  First  the 
real  world  produces  an  immediate  perceptual  response.  The  adversary  reacts  to 
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the  percs tved  environment  specifically  by  noting  needs,  reacting  to  its  own 
previous  actions  (e.g.,  a  previous  failure  to  satisfy  a  need  might  provide  the 
stimulus  for  subsequent  action),  the  actions  of  opponents  to  block  or  facilitate 
need  satisfaction,  the  capabilities  of  the  adversary  and  of  opponents  to  satisfy 
needs  through  actions.  In  this  first  step  the  adversary  is  selecting  aspects  of 
the  environment  to  attend  (e.g.,  what  is  the  need,  what  is  the  problem),  coding 
those  aspects  along  important  dimensions,  and  generally  character iz ing  the 
nature  of  the  environment  Into  those  parts  that  require  action  or  attention 
(i.e.,  identifying  needs),  and  those  that  do  not. 

The  second  step  of  the  intention  process  involves  the  organisation  of 
perceptions,  estimates,  and  expectations  into  orderly  categories  of  Information. 
Among  these  categories  are  the  following:  estimates  of  current  and  future  re¬ 
sources,  both  for  one's  own  side  and  for  the  oppos  I  te  s  I  de,  estimates  of  pos¬ 
sible  actfons  on  each  side  and  the  risks  attached  to  them  and  also  the  utilities 
attached  to  the  outcome  of  the  varfous  actions.  The  latter  Information  requires 
the  analyst  to  form  some  Impression  of  the  adversary's  value  system.  The  ana¬ 
lyst  needs  to  estimate  how  the  adversary  evaluated  outcomes  for  each  side.  It 
Is  helpful  to  separate  the  elements  of  this  step  Into  those  that  deal  with 
Issues  of  fact  (which  can  also  be  thought  of  as  predictions)  and  those  that  deal 
with  Issues  of  value  and  evaluation.  For  example,  the  enemy  may  be  contempla¬ 
ting  five  different  modes  of  attack.  Several  Issues  of  fact  must  be  estimated: 
what  resources  are  needed  for  each  mode,  what  is  the  I  Ikel  Ihood  of  each  mode 
succeeding,  what  is  the  opponent  capable  of  doing  In  reaction  to  each  mode  and 
what  are  the  probabilities  of  each  of  those  reactions.  Additionally,  various 
I ssues  of  val ue  must  be  assessed:  how  willing  Is  the  adversary  to  expend  re¬ 
sources  at  various  levels  (I.e.,  what  is  the  cost  of  the  resources),  how  good 
are  the  various  possible  outcomes  of  the  different  attack  modes  (I.e.,  what  are 
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the  benefits)  and  what  are  the  values  to  the  parties  on  both  sides  of  the 
enemy's  reactions  to  the  various  possible  models.  To  make  these  assessments  the 
analyst  must  form  some  Impression  of  how  the  adversary  makes  estimates  of  facts 
and  probabilities  and  how  he  assesses  values,  both  for  the  adversary's  own  ac¬ 
tions  and  for  his  estimates  of  the  analysts’  country’s  reactions  and  initia¬ 
tives. 

The  third  step  in  the  Intention  process  integrates  the  Information  on  facts 
and  values  to  determine  what  actions  would  be  better  than  other  actions  and  the 
I  ike  I  I  hood  of  various  outcomes  and  benef  I  ts.  Poss  I  b  I  e  goa  Is  are  then  ranked 
according  to  value,  risk,  probability,  cos^,  or  any  other  criteria  of  importance 
to  the  adversary  decision-making  system.  In  this  step,  the  adversary  Is 
formulating  expectations  and  clarifying  those  "mental  Images  of  future  events" 
which  might  be  brought  to  reality. 

To  assess  this  step  In  the  intention  process  the  analyst  may  have  to 
consider  the  adversary's  decision-making  rules.  For  example,  suppose  one  mode 
of  attack  offers  the  adversary  low  risks  of  loss  but  little  in  the  way  of 
tact tea I  advantage,  i.e.,  low  gains.  Another  mode  of  attack  offers  spectacular 
gains,  but  at  the  possible  risk  of  huge  losses.  If  the  adversary  allows  these 
dimensions  of  risk  and  gain  to  compensate  for  each  other  (i.e.,  a  lot  of  gain 
makes  up  for  a  lot  of  risk)  the  riskier,  more  advantageous  course  might  receive 
the  higher  ranking.  However,  If  the  adversary  cannot  affort  such  compensating 
trade-offs  (e.g.,  because  resour.es  are  so  limited  that  any  major  risk  of  loss 
is  intolerable)  then  an  unacceptable  state  on  a  single  dimension,  such  as  risk, 
may  be  sufficient  to  eliminate  that  option  from  further  consideration. 
Obviously,  a  similar  conclusion  might  be  reached  by  the  analyst  who  attends  to 
whether  the  adversary  is  generally  a  gambler  or  conservative.  The  decision 
analysis  approach  has  the  added  advantage  that  It  attempts  to  identify  why  the 
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adversary  might  be  conservative  or  Intrepid. 

One  aspect  of  the  adversary  which  may  play  an  Important  role  at  this  step 
Is  doctrine,  those  prescriptions  as  to  what  types  of  action  outcomes  should  be 
highly  valued  (e.g.,  sleze  the  h I gh  ground)  or  avol  ded  (e.g.,  never  concentrate 
In  range  of  enemy  guns).  Doctrine  can  be  viewed  as  codified  decisions,  or 
stereotyped  Integrations  of  estimates  and  assessments  from  past  experience. 
Such  codes  may  offer  Important  clues  as  to  how  the  adversary  will  integrate  the 
estimates  and  assessments  bearing  on  the  current  problem  It  Is  always  possible, 
however,  that  the  adversary  will  flout  doctrine,  either  Intentionally  (e.g.,  as 
an  explicit  means  to  achieve  surprise),  or  unintentionally  (e.g.,  out  of 
Ignorance  or  confusion).  It  Is  also  possible  for  doctrine,  when  appl led  to  a 
specific  detailed  case,  to  yield  Incompatible  recommendations  (e.g.,  the 
principles  of  war  Include  both  "concentration"  and  "economy  of  force,"  two 
recommendations  that  are  rarely  simultaneously  compatible),  and  thus  offering 
little  guidance  to  either  the  adversary  decision-maker  or  the  Intention 
estimating  analyst. 

In  the  fourth  step  the  adversary  selects  a  subset  of  goals  from  the  rank¬ 
ings  created  in  the  third  step.  This  constitutes  the  Intention  decision,  or  the 
adversary's  pol  Icy.  The  adversary  selects  a  desired  and  possible  outcome  (or 
objective)  based  on  the  benefits  that  would  seemingly  result  from  It  and  the 
apparent  costs  of  the  path  to  that  outcome,  given  the  resources  that  are 
aval  table  for  accompl  Ishlng  that  outcome,  the  feasible  actions  that  are  re¬ 
quired,  and  the  anticipated  contingencies  of  enemy  response.  The  steps  preced¬ 
ing  this  one  Involve  estimating  these  aspects  of  the  decision.  The  steps  subse¬ 
quent  to  this  one  entail  refinement  of  the  policy  selection  made  In  this  step. 

If  the  challenge  to  the  analyst  In  the  previous  step  was  to  narrow  down  the 
range  of  possible  courses  of  action  to  those  the  adversary  would  consider 
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feasible  and  worthwhile,  the  challenge  In  this  step  Is  to  determine  which 
objective  or  policy  the  adversary  will  value  most  highly.  An  analysis  of 
successful  I ntentlon  esti  mat  ion  by  I  ntel I  I gence  analysts  In  the  past  (Stech, 
1979)  reflects  two  characteristics  that  might  aid  other  analysts. 

Successful  Intention  estimators  used  rather  explicit  analytical  models  of 
hew  the  adversary  made  decisions.  These  models  related  the  general  war  objec¬ 
tives  of  the  adversary  (the  two  historical  cases  took  place  In  World  War  1 1 )  to 
the  adversary's  resources  and  constraints.  These  explicit  models  were  refined 
by  a  process  of  Inference).  That  I  s,  the  ana  I  ysts  determ  I  ned  how  ev  I  dence  of 
different  kinds  would  lead  them  to  change  the  models.  The  analysts  then  sought 
the  necessary  evidence  to  determine  which  direction  the  models  should  evolve. 
In  effect,  the  analyst  anticipated  the  possible  behaviors  of  the  adversary  and 
considered  the  Implication  of  each  possible  move  for  the  evolving  model  of  the 
adversary's  Intentions.  As  the  evidence  became  available,  the  analyst  was  able 
to  refine  the  model,  making  It  more  explicit.  Eventually,  the  enemy's  actions 
and  reactions  could  be  estimated  with  considerable  accuracy.  This  process  by 
the  analysts  was  one  of  "sagacity,"  I.e.,  understanding  how  the  adversary  had 
behaved  In  the  past  and  the  nature  of  his  motives  and  "acumen,"  I.e.,  the 
analyst's  ability  to  understand  and  anticipate  the  enemy's  response  and  to 
duplicate  the  mental  decision-making  logic  of  the  adversary  (see  Stech,  1981, 
ch.  2,  on  sagacity  and  acumen).  To  a  lesser  degree  the  analysts  were  able  to 
use  "control"  to  assist  In  the  development  of  their  models.  That  is,  the 
analysts  were  at  times  able  to  control  the  Information  or  the  situation  the 
adversary  faced  (e.g.,  by  sending  false  Information  through  double  agents  or 
deception  operations,  or  by  knowing  In  advance  what  one's  own  forces  were  plan¬ 
ning  to  do)  and  could  thus  use  this  control  to  settle  open  questions  about  the 
enemy's  reactions  and  decisions. 
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The  fifth  step  In  the  intention  process  consists  of  translating  the  Inten¬ 
tion  decision,  or  policy.  Into  a  strategic  concept  with  a  general  objective  and 
general  courses  of  action  outlined.  This  outline  forms  the  basis  for  subsequent 
detailed  planning.  It  Identifies  what  the  adversary  Intends  to  achieve  and  h£&L 
this  will  be  accomplished  to  a  degree  sufficiently  detailed  so  that  planners  can 
turn  this  strategic  concept  Into  concrete  actions. 

The  sixth  and  seventh  steps  involve  detailed  planning  so  that  the  intention 
can  be  carried  out  in  action.  First  (step  six)  planners  must  determine  from  the 
strategic  concept  what  specific  actions  and  subobjectives  are  required  to  attain 
the  overal I  strategic  objective.  The  necessary  steps  must  be  ordered  Into  a 
sequence  of  operations  so  that  each  sub-step  is  completed  before  subsequent 
steps  that  build  on  It  are  undertaken.  Enemy  actions  have  to  be  anticipated  and 
losses  calculated.  Orders  for  required  resources  are  Issued.  Warnings  are 
given  to  troop  and  naval  untls.  Training  operations  focus  on  anticipated  mis¬ 
sions.  Studies  are  made  of  uncertain  aspects  of  the  plan.  Reconnaissance  of 
the  target  area  is  Increased.  Deception  operations  may  be  undertaken  to  conceal 
these  detailed  preparations. 

In  the  last  step  of  the  Intention  process,  the  adversary  writes  and  Issues 
the  detailed  plans  for  the  intended  operation.  This  may  have  the  traditional 
form  of  the  military  or  naval  operational  order,  or  It  may  be  far  more  Informal 
and  instantaneous,  for  example,  it  may  simply  consist  of  the  codeword  that 
signals  the  execution  of  an  operation  that  was  planned  long  before.  In  this 
step,  those  actors  that  must  carry  out  the  adversary’s  Intentions  must  carry  out 
the  final  steps  before  actual  action  Itself:  assigning  responsibilities,  speci¬ 
fying  subobjectives,  allocating  resources,  issuing  maps  and  plans,  arming 
forces,  fueling  vehicles,  sending  out  scouts,  etc.  The  adversary  has  not  com- 
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miffed  himself  fo  fhe  plan,  buf  Is  ready  fo  do  so  and  has  faken  ai  I  buf  fhe 
final  sfep  of  acflon  Ifself. 

The  descrlpflon  of  fhese  seven  sfeps  In  fhe  Intenflon  process  makes  If 
clear  fhaf  fhe  adversary’s  behavior  becomes  easier  fo  monlfor  as  fhe  Infenflon 
process  approaches  fhe  acflon  sfep.  More  physical  acflvtfles  and  preparations 
are  needed  to  translate  the  strategic  goal  Into  strategic  technique  and  then 
Into  tactical  plans  then  are  needed  at  earlier  stages.  Thus  the  cioser  the 
adversary  comes  to  actually  carrying  out  his  Intentions,  the  greater  the  amount 
of  physical  Intel  I Igence  the  adversary  will  be  forced  to  generate  and,  I  f  he 
wishes  to  conceal  his  Intentions,  to  hide  or  disguise.  (Sut  hiding  and 
disguising  are  also  actions  which  generate  physical  intelligence.) 

Oavld  Kahn  (1978:  39-41)  makes  a  useful  distinction  between  "physical 
Intelligence"  and  "verbal  Intelligence."  Physical  Intelligence  is  derived  from 
the  natural  resources,  physical  Instal latlons,  numbers  of  weapons  and  troops 
available,  vol ume  of  commercl al  trade,  and  so  forth.  Verbal  I ntel I Igence  I  s 
derived  from  words  through  plans,  orders,  morale,  perceptions.  Intentions, 
estimates,  promises,  or  motives. 

Plans  and  Intentions  take  time  to  translate  Into  physical  reality.  Knowing 
an  adversary's  Intentions  gives  time  to  react,  whereas  knowing  what  actions  the 
adversary  has  already  begun  to  take  may  leave  I Ittle  or  no  time  for  reaction. 
While  the  latter  steps  of  the  intention  process  generate  more  physical 
Intelligence  than  the  earlier  steps,  the  earl ler  steps  may  tend  to  generate  more 
verbal  Intelligence,  I.e.,  Information  about  how  the  adversary  perceives,  esti¬ 
mates,  makes  Judgments  and  declsons,  and  so  on.  In  modeling  the  Intention 
process  of  the  adversary,  the  analyst  will  have  to  use  verbal  Intelligence  and 
physical  Intelligence  to  Infer  the  events  between  the  real  world  situation  and 
the  adversary's  act  tons. 
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E.  CONTENT  OF  ANALYTICAL  METHOOS 


Intentions  with  certain  characteristics  are  easily  predicted.  To  the 
degree  that  he  analyst  can  determine  that  the  Intentions  are  of  the  more 
predictable  type  he  can  put  greater  confidence  In  his  estimates. 

Stable  Intentions  are  predictable.  Two  types  of  stability  apply  to  Inten¬ 
tions.  Some  behaviors  are  habitual,  t.e.,  prompted  by  recurring  needs  and 
drives.  The  need  of  a  nation’s  people  for  food  and  shelter  during  a  crisis  such 
as  war  may  lead  to  an  I ntentlon  by  the  nation’s  I  eadersh  I  p  to  support  minimal 
living  standards.  Other  behaviors  are  customary  or  respond  to  strong  cultural 
norms  and  are  thus  stable  so  long  as  these  norms  remain  effective.  Habitual  and 
customary  behaviors  are  highly  predictable.  If  the  analyst  develops  the  ability 
to  recognize  the  repetitive  pattern  and  the  stimuli  that  summon  such  behaviors. 
In  the  presence  of  such  conformity  Inducing  situations  the  adversary  will  tend 
to  demonstrate  the  same  habits  or  customs.  In  these  eliciting  situations  the 
adversary’s  behavior  Is  highly  autocorrelated,  and  shows  low  variance  over  these 
situations.  Habits  are  reliable  reactions  to  aroused  needs  and  customs  are 
reliable  reactions  In  response  to  a  stable  intention  to  conform.  The  main  task 
of  the  analyst  Is  to  recognize  the  autocorrelated,  rel lable  behaviors  and  then 
determine  what  needs,  situations  or  events  trigger  these  stable  responses. 

A  second  form  of  stable  Intention  occurs  when  a  form  of  behavior  Is  con¬ 
stantly  repeated  over  time  regardless  of  particular  situations.  These  behaviors 
have  very  high  base  rates  and  will  appear  to  be  normal,  routine  behavior.  These 
behaviors  may  serve  some  functional  purpose  (e.g.,  regular  patrolling  and  recon¬ 
naissance  of  sensitive  terrain)  or  may  be  merely  a  tradition  serving  only  cere¬ 
monial  purposes  (May  Oay  parades).  Some  behavior  may  lack  any  recognized  tradi¬ 
tion  and  may  be  repeated  merely  because  no  alternative  occurs  to  those  who 


17 


perform  the  behavior.  An  analyst  may  create  a  plausible  explanation  for  such 
behavior,  but  Its  actual  "cause”  may  be  simply  that  "It  was  always  done  that 
way,"  I.e.,  mere  behavioral  Inertia.  It  takes  effort  to  change  things,  and 
there  Is  no  point  to  fixing  something  that  Is  not  broken.  Similarly,  behaviors 
that  cause  no  harm  may  be  tolerated  and  even  reinforced  although  they  produce  no 
benefits  except  to  keep  people  busy.  Behavior  that  has  been  repeated  under  a 
wide  variety  of  situations  and  circumstances  with  little  variation  In  character 
Is  likely  to  continue.  The  Intention  for  such  behavior  may  relate  to  Identifi¬ 
able  goals  or  It  may  be  simply  the  Intention  that  what  was  done  be  done  again. 
High  base  rate  behavior  Is  highly  predictable.  Behavior  that  conflicts  with  or 
prevents  high  base  rate  behavior  will  require  strong  motives  and  Intentions  to 
overcome  the  Inertia  of  the  repeated  behavior. 

Some  situations  bring  a  multitude  of  pressures  to  bear  on  a  nation,  al I 
acting  to  motivate  behavior  In  a  certain  direction.  For  example,  a  massive 
sudden  surprise  attack  will  mobf  I  tze  a  nation's  defenses.  To  the  degree  that 
the  forces  and  pressures  on  the  attacked  nation  tend  to  work  In  the  same  direc¬ 
tion,  the  nation's  behavior  and  Intentions  are  more  predictable.  For  ex'y/'.-. 
Imagine  a  small  nation,  unsure  of  Its  survival,  with  many  enemies  e---*  ; 
allies,  vulnerable  borders  and  trade  routes,  a  militaristic  tradition,  capable 
standing  armed  forces,  a  unified  and  resourceful  population,  a  tradition  of 
antipathy  toward  the  attacker,  an  economic,  political,  or  territorial  Interest 
In  the  outcome  of  war,  and  a  history  of  racial  or  re!  Iglous  disputes  w Ith  the 
attacker.  Such  a  nation  Is  more  predictably  going  to  respond  mi  I  Itarl  ly  to  a 
surprise  attack  than  a  nation  with  opposite  characteristics  and  pressures.  Many 
predisposing  forces  and  pressures  acting  In  a  given  direction  allow  a  more 
confident  prediction  or  estimate  of  Intentions  then  cross-cutt I ng  and  conflict¬ 
ing  pressures.  In  other  words,  when  a  nation  faces  intense  demands,  pressures. 
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and  forces  to  act  tn  a  certain  way,  and  little  or  no  opposing  pressures  to  such 
action,  It  Is  likely  to  respond  to  the  pressures.  The  more  pressures  that  act 
In  the  same  direction,  the  easier  the  task  of  forecasting  the  nation's  Inten¬ 
tions. 

There  Is  a  nontrivial  danger  In  using  a  causa!  analysis  of  the  converging 
pressures  on  a  nation  to  estimate  Its  Intentions,  however.  What  may  appear  to 
the  analyst  to  be  an  Irreslstable  pressure  might  In  facl"  pose  only  a  temporary 
problem  to  the  adversary  nation.  On  the  other  hand,  what  appears  to  be  a 
trivial  factor  to  the  analyst  might  be  an  overwhelming  demand  that  the  leaders 
of  the  adversary  state  cannot  Ignore.  An  example  of  the  former  case  might  be 
Egypt  prior  ic  the  1973  attack  on  Israel.  Egypt  seemingly  faced  severe  limita¬ 
tions  of  mill tary  caps* 1 1  I ty  which  wou I d  prevent  any  successful  attack.  I  n 
fact,  Egypt  had  evolved  alternative  solutions  to  Its  military  limitations  (o.g„ 
Ingenious  means  for  rapidly  cutting  through  the  sand  dunes  along  the  Suez  Canal 
with  high  pressure  fire  hoses)  and  of  adjusting  Its  military  Intentions  to  Its 
military  capabilities  so  as  to  accomplish  overriding  political  and  diplomatic 
goals.  An  example  cf  the  latter  case  might  be  the  underestimation  by  Western 
analysts  prior  to  1941  of  the  criticality  of  resources  to  Japan  and  the  centra! 
role  of  qN  supplies  to  Japanese  intentions.  The  Western  embargo  on  oil  sales, 
an  attempt  to  use  economic  force  to  pressure  Japan  to  moderate  Its  China  policy, 
had  the  effect  of  Increasing  Japan's  militaristic  ambitions  (to  secure  reliable 
oil  supplies  In  Southeast  Asia)  rather  than  curtailing  them. 

Just  as  there  are  behaviors  that  are  compel  led  by  converging  pressures, 
there  are  other  behaviors  that  require  converging  preparations,  i.e.,  complex 
and  multifaceted  multiple  actions  that  lead  up  to  and  prepare  for  some  behavior 
the  state  Intends  to  act  out.  Such  preparations  are  like  a  pitcher's  wind-up, 
they  signal  an  oncoming  event,  even  ^ hough  they  may  not  reveal  the  exact  nature 
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of  the  pitch  that  Is  thrown  or  the  base  the  pitcher  may  throw  to.  The  enTlre 
complex  of  preparations  may  suggest  one  single  Integrared  intention,  or  may  be 
associated  with  disparate,  disconnected  events.  The  chronology  of  observations 
(by  the  Intention  estimating  analysts)  may  or  may  not  reflect  the  logical  order 
of  acts  dictated  by  a  single  plan;  Instead  the  observational  chronology  may 
result  from  the  mxlng  of  actions  called  for  by  several  unconnected  plans.  On  the 
other  hand,  certain  plans  and  operations  require  that  various  essential 
preparations  be  made  In  predictable  order.  If  a  watch  Is  set  for  these 
preparations  and  none  are  detected,  this  Is  a  negative  Indicator;  It  Is  unlikely 
the  adversary  Is  preparing  that  particular  operation.  Of  couse,  the  adversary 
may  anticipate  such  a  watch  and  disguise  such  preparations  or  cover  them  with  a 
plausible  deception.  If  both  predisposing  pressures  and  necessary  preparations 
for  a  particular  behavior  are  absent,  there  Is  little  likelihood  that  the 
adversary  ■  Intends  to  perform  that  behavior. 

Certain  planned  actions  by  an  adversary  require  various  acts  be  taken  or 
processes  be  set  In  motion  to  carry  out  the  Intention.  The  attack  on  Pearl 
Harbor  for  example,  required  that  the  Japanese  fleet  sail  to  Hawaii.  These  acts 
may  partially  determine  the  courses  of  action  open  to  the  adversary.  While  at 
some  point  on  Its  rou  T9  to  Hawaii  the  Japanese  fleet  could  have  aborted  the 
attack  (e.g.,  If  It  were  discovered  by  U.S.  reconnaissance),  there  was  a 
particular  moment  at  which  there  could  be  no  turning  back  —  the  air  raid  had  to 
be  launched  regardless  of  whatever  else  was  taking  place. 

Some  plans  by  an  adversary  may  be  sufficient  determined  by  their  early 
events  that  the  conclusions  can  be  predicted.  The  analyst  observing  the 
unfolding  of  the  plan  may  be  able  to  anticipate  the  ultimate  target  by  noting 
that  certain  options  and  directions  have  been  closed  off  as  the  operation  has 
progressed.  Indicating  the  Intended  direction  of  the  adversary's  actions. 
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To  the  extent  that  the  adversary's  behavior  Is  under  the  control  of  some 
agency  that  the  analyst  Is  able  to  monitor,  the  behavior  I  s- more  predictable. 
For  example.  In  combat  the  analyst's  own  nation  will  conduct  operations  which 
may  determine  the  adversary's  reactions.  If  the  analyst  knows  of  the  Impending 
operations  against  the  adversary,  he  or  she  can  anticipate  the  impact  on  the 
adversary's  response.  The  analyst's  nation  may  be  able  to  control  the  adver¬ 
sary's  access  to  Information,  as  for  example  In  Britain  In  World  War  il  when  all 
Nazi  esp l onage *  agents  were  captured  and  controlled  by  British  security  forces. 
This  control  of  Information  can  allow  the  analyst  to  better  model  the  adver¬ 
sary's  decision  process  by  providing  direct  access  to  some  of  the  Information 
that  goes  Into  that  process. 

An  example  of  how  Information  control  assisted  Intel  1  tgence  analysts  to 
estimate  Intentions  occurred  In  the  Pacific  In  World  War  II  prior  to  the  battle 
of  Midway.  Through  code-break  I ng,  U.S.  naval  Intelligence  was  able  to  determine 
that  the  Japanese  intended  to  launch  a  major  attack  to  destroy  the  few  remaining 
U.S.  aircraft  carriers.  The  Japanese  Intended  to  draw  on  the  U.S.  carriers  by 
first  attacking  a  kev  U.S.  land  base.  The  question  unanswered  by  the  code- 
breeklng  was  where  this  land  attack  was  to  take  places  Midway,  Oahu,  the 
Aleutians,  the  U.S.  west  coast  were  al  I  posslbl  l  itles  with  Midway  being  the 
prime  candidate.  To  conffrm  that  the  Japanese  Intended  to  attack  Midway,  U.S. 
Navey  Intelligence  conducted  an  "Intelligence  experiment,"  leaking  the 
Information  that  Midway  Island  was  short  of  water.  This  was  picked  up  by  Jap¬ 
anese  I  Istenlng  posts  which  reported  to  Tokyo  That  the  target  for  the  forth¬ 
coming  attack  was  having  water  problems.  These  signals  were  intercepted  and 
reed  by  the  U.S.  Navy  confirming  the  suspicion  that  the  Japanese  were  aiming  at 
Midway.  The  successful  trap  sat  by  the  U.S.  Navy  for  the  Japanese  depended  on 
this  confirmation  by  Information  control. 
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Analyst  Logic  and  Reason  1  no  Processes 


It  is  possible  to  separate  Intention  estimation  Into  primary  steps  of 
estimation  logic,  and  then  determine  where  parts  of  the  process  might  be  strong 
or  weak.  For  this  we  divide  estimation  Into  seven  different  steps  needed  to 
reach  an  analytic  conclusion  (see  Figure  3).  These  steps  are  somewhat  arbitrary 
and  could  be  combined  Into  fewer  or  expanded  Into  more  steps.  Nonetheless,  they 
seem  characteristic  of  Inference  and  deductive  reasoning  as  studies  by  cognitive 
psycho  I  og  I  sts  and  seem  to  capture  the  var  I  ous  capab  II  I  tl  es  of  Individuals  to 
process  Information.  All  Intention  estimation  Involves  at  some  point  The  think¬ 
ing  of  Individual  analysts  so  an  appraisal  of  the  analytic  logic  needed  to  make 
estimates  should  serve  to  reveal  some  of  the  noncbvlous  perennial  difficulties 
of  Intention  estimation. 

The  mental  methods  people  use  to  process  Information  have  been  labeled 
"heuristics1'.  These  are  usually  adequate  for  the  reasoning  we  typically  need  to 
do.  Certain  problems  go  beyond  the  capabilities  of  these  heuristics,  however, 
and  require  more  elaborate,  less  speedy,  and  less  efficient  methods  than 
everyday  heuristics.  Often  It  Is  difficult  to  recognize  that  the  everyday 
heuristics  of  thinking  are  inadequate  and  people  continue  to  use  them  when  they 
should  use  stronger  reasoning  methods. 

Eer.cfti.Waa,  flata 

One  such  heuristic,  "aval  I abll  ?ty",  Is  used  when  people  perceive  and  encode 
data  (see  Ffgura  4).  An  analyst  often  decides  whether  an  evsnt  has  often 
occurred,  and  Is  thus  typical,  or  whether  It  Is  rare  and  unusual  behavior. 
Various  character! sties  of  events  Influence  how  memorable  they  are;  the 
frequency  of  their  occurrence,  the  sal lenca  to  tha  analyst  of  the  event.  Its 
vividness.  Frequent,  salient,  vivid  events  are  more  memorable.  Memorability  In 


Figure  3.  Steps  in  Estimation  Logic 


o  Perceiving  Data 
o  Weighting  Data 
o  Characterizing  Data 
o  Assessing  Covariations 
o  Assessing  Causes  and  Effects 
o  Prediction 
o  Theorizing 


Figure  4.  Perceiving  Data:  Availability 
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turn  Influences  judgements  of  frequency  and  likelihood.  The  events  we  remember 
best  we  tend  to  believe  are  more  likely  or  more  frequent.  Note,  however,  that 
we  can  believe  an  event  Is  frequent  even  If  frequency  were  not  the 
characteristic  of  the  event  that  led  to  It  being  memorable.  We  may  perceive  a 
vivid  or  salient  event  as  frequent,  even  when  it  Is  not,  because  we  remember  It 
well.  Or  to  put  It  the  other  way,  we  might  remember  something  very  well  because 
It  was  sal  lent  or  vivid,  and  then,  because  it  was  memorable,  estimate  that  It 
was  frequent  or  likely.  The  more  events  are  available  to  memory,  for  whatever 
reason,  the  more  we  tend  to  overestimate  their  frequency  and  likelihood. 
Uncommon  events  may  be  perceived  as  common. 

In  Intention  estimation,  character  I st ics  of  events  other  than  their 
frequency  or  typicality  seem  to  enhance  their  memorability  and.  In  turn,  inflate 
analysts'  estimates  that  these  are  typical  behaviors.  International  crises  are 
highly  salient  and  vivid  to  the  Intelligence  analysis  community  and  very 
memors,  le.  It  Is  I  Ikely  that  analysts  tend  somewhat  to  overestimate  the  I  Ikel  I- 
hood  that  an  adversary  will  shift  from  noncrisis  to  crisis  behavior  In  periods 
of  tension.  The  "false  alarm"  rate  for  predictions  of  adversary  crisis  action 
will  thus  tend  to  be  high. 

The  proximity  of  one  event  In  time  or  space  to  another  event  a  I  lows  the  two 
to  be  more  easily  remembered  as  connected.  Such  memorability  of  a  connection 
may  lead  analysts  to  perceive  coincidental  events  as  causally  related.  One 
example  seems  to  have  occurred  in  the  case  of  the  1969  Libyan  coup  and  the  1973 
Cold  War.  The  mere  presence  of  Soviet  Navy  ships  plus  these  two  events 
seemingly  led  analysts  to  suspect  causal  connections  between  the  events  and  the 
Soviets  which  better  evidence  tends  not  to  support. 

Causal  links  between  events  are  among  the  most  memorable  data  characterise 
tics.  To  the  degree  that  an  analyst  perceives  a  causal  link  between  an  adver- 
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sary's  behavior  and  U.S.  behavior,  not  only  Is  the  causal  link  likely  to  be 
remembered,  but  there  Is  the  risk  that  this  memorability  will  lead  the  analyst 
to  overestimate  the  frequency  with  which  such  causal  Influence  takes  place  or  Is 
Itkeiy  In  the  future.  Causal  links  are  difficult  or  impossible  to  prove,  how¬ 
ever,  but  are  readily  perceived,  even  when  they  don't  exist,  as  Is  discussed 
further  below.  So  the  combination  of  a  vivid,  memorable,  but  spurious,  causal 
link,  and  the  availability  heuristic,  could  lead  an  analyst  to  overestimate  the 
likelihood  of  a  causal  relationship  that.  In  fact,  does  not  really  exist,  or  Is 
extremely  rare.  Again,  false  alarms  about  what  the  adversary  Is  able  to  cause, 
or  Intends  to  cause,  are  likely. 

A  second  heuristic  which  affects  the  way  analysts  perceive  data  Is 
"representat I veness"  (see  Figure  5).  When  the  analyst  assesses  an  event, 
certain  characteristics  and  features  of  the  event  lead  him  to  Infer  the  event  Is 
of  one  type  rather  than  another,  and  to  estimate  what  population  of  events  this 
partlcu I  ar  one  came  from  (e.g.,  Is  this  partlcu I  ar  behavior  "host 1 1 e"  or  Just 
"unfriendly"?).  There  Is  nothing  wrong  with  this  logic  li  the  analyst  gives  at 
I  east  some  thought  to  the  popu  I  at  ion  pool  I  tsel  f ;  ask  I  ng  If  It  Is  likely  that 
any  event,  regardless  of  Its  characteristics,  came  from  that  particular  pool  of 
behavior  (e.g.,  In  general,  Is  the  adversary's  behavior  more  often  "hostile"  or 
"unfriendly"?).  Some  behavior  populations  are  highly  Improbable  (e.g.,  the 
launching  of  surprise  attacks)  making  It  unlikely  that  a  given  event  under  study 
came  from  that  population,  no  matter  how  strong  the  resemblance  might  be  between 
the  features  of  the  particular  event  and  the  character  of  that  population  of 
events.  (That  Is,  many  more  behaviors  may  look  I  Ike  preparations  for  surprise 
attacks  than  actually  are  preparations  for  surprise  attacks.)  Unless  the 
analyst  considers  the  base  rate  frequency  of  the  population  of  behaviors  as  well 
as  considering  the  features  of  the  specific  event,  there  Is  the  possibility  he 
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or  she  will  overestimate  atypical  and  unusual  events  and  perceive  them  as  more 
common  than  they  really  are. 

Analysts  who  attend  to  the  doctrinal  statements  of  an  adversary  may  find  a 
statement  or  set  of  pronouncements  that  suggest  a  particular  future  Intention. 
The  specific  character  I st ics  of  these  statements  may  be  very  suggestive  of  a 
future  action,  e.g.,  open  hostility  with  other  countries.  These  features  of 
these  particular  statements  have  to  be  balanced  against  the  base  rate  of 
analogous  statements  and  the  base  rate  of  behavior.  If  the  entire  body  of 
analogous  statements  reflects  a  peaceful  outlook,  or  a  cautious  attitude,  the 
analyst  should  reduce  the  weight  given  to  the  hostile  remarks.  Similarly,  If 
the  nation  being  monitored  has  had  a  history  of  peaceful  relations  the  analyst 
should  consider  the  hostile  statements  of  representing  an  unusual  departure  and 
a  possible  anomaly  rather  than  a  clear-cut  Indicator  of  the  future. 

Similarly,  analysts  often  make  major  deductions  about  an  adversary's 
tactics  or  strategy  using  military  equipment  construction  as  evidence.  That  Is, 
the  construction  of  a  particular  weapon  Is  taken  as  evidence  of  a  particular 
Intention.  Such  arguments  may  lose  sight  of  the  other  naval  or  military 
construction  programs  which  may  represent  even  greater  emphasis  or  effort.  The 
use  of  naval  events,  for  example,  as  evidence  of  naval  pol Icy  or  doctrine  may 
fall  victim  to  the  same  problems,  that  Is,  using  rare  and  unusual  events  to  draw 
conclusions  about  typical  or  general  naval  behavior. 

A  fascinating  example  of  the  tenuousness  of  estimates  of  enemy  Intentions 
extrapolated  from  estimates  of  enemy  military  capabilities  Is  given  In 
McLachlan's  (1968)  account  of  British  Naval  Intelligence  before  and  during  world 
War  II.  In  1936,  when  the  Germans  were  constructing  their  great  battleship 
B 1  saark.  the  consensus  of  Br I tl  sh  dl pi omat I c  opinion  was  that  Germany  would 
adhere  to  the  Anglo-German  Naval  Treaty  of  1936,  which  limited  German  battleship 
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size  to  35,000  tons.  This  consensus  provl  ded  the  basis  for  the  design  of  the 
8rltlsh  battleship  KI nq  George  V.  The  dimensions  of  the  Blsmark  and  T1 rpltz  as 
released  by  the  Germans  Indicated  that  If  these  ships  did  In  fact  displace  only 
35,000  tons  as  the  Germans  claimed,  they  were  of  much  shallower  draft  than  the 
British  shtp.  Although  the  lower  tonnage  was  doubted  by  Naval  Intel  I  Igence 
nontechnical  officers,  Intel  I Igence  opinion  was  divided.  From  the  Inferred 
characteristics  (shallow  draft  and  35,000  tons)  The  British  Naval  Plans  Olvlsion 
concluded  "the  present  delsgn  of  German  capital  ships  appears  to  show  that 
Germany  Is  looking  towards  the  Baltic  with  Its  shallow  approaches  more  than  In 
the  past"  (quoted  by  McLachlan,  1968:  136),  that  Is,  the  German  ships  were  aimed 
more  at  Russia  than  at  Britain.  In  fact,  Blsmark  and  TTrp Itz  were  designed  to 
be  roughly  45,000  tons,  and  the  Germans  released  false  figures  to  the  British, 
depending  on  Br  I  tl  sh  belief  I  n  the  read  I  ness  of  the  Germans  to  honor  the  1 936 
agreement.  The  Germans  read  their  opponent  wollj  the  British  Director  of  Plans 
at  the  time  wrote,  "our  principal  safeguard  against  such  an  Infraction  of  treaty 
obi  I  gat  Ions  lies  In  the  good  faith  of  the  signatories"  (p.  137  In  McLachlan). 
Not  only  had  Germany  deceived  Britain  as  to  her  capabilities,  she  had  the 
additional ,  un I ntended  benef  1 1  of  an  erroneous  British  estl mate  of  Germany's 
naval  Intentions.  Similar  underestimates  based  on  similar  German  deceptions 
regarding  submarines,  cruisers,  and  battlecruisers  also  occurred  prior  to  World 
War  II  and  may  also  have  misled  British  estimates  of  Germany's  willingness  to 
engage  British  naval  power  wtth  what  the  British  took  to  be  a  far  less  capable 
navy  than  Germany  In  fact  possessed. 

Similarly,  U.S.  un  erestlmates  of  the  range  end  performance  of  the  Japanese 
Zero  and  the  estimate  that  shallow  water  torpedo  attacks  were  tnfeaslble  prob¬ 
ably  Contributed  to  the  Japanese  surprise  In  attacking  Pearl  Harbor  (Woh I  setter, 
1962:  394). 
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Fear  of  the  Luftwaffe's  strategic  bombing  capability,  overestimated  in  1938 
by  the  British  Air  Staff,  Inflated  British  Cabinet  estimates  of  Germany's 
willingness  to  go  to  war  over  the  Czechoslovakia  crisis  and  contributed  to 
Chamberlain's  motivation  to  capitulate  to  Hitter,  according  to  some  historians 
(see  Bracken,  1977). 

Finally,  it  is  quite  easy  to  perceive  plausible  --duses  for  almost  any 
event,  and  to  detect  causes  that  have  no  actual  connections  to  events,  and  which 
may  be  quite  rare  and  atypical.  Features  of  a  data  sample  may  lead  the  analyst 
to  perceive  a  causal  relationship,  but  It  is  Important  that  analysts  stop  to  ask 
if  such  relations  are  generally  likely  or  typical.  If  not,  the  analyst  should 
lower  his  estimate  that  such  unusual  causal  behavior  Is  occuring  In  the  case 
under  study. 

Weighting  Data 

Not  only  must  analysts  perceive  data,  they  must  weight  it,  deciding  which 
pieces  of  information  are  important  and  which  are  less  critical  to  their  judg¬ 
ments  (see  Figure  6).  However,  psychologists  have  found  that.  In  general, 
people  are  not  at  all  accurate  in  identifying  what  Information  they  actually  use 
to  make  Judgments  and  decisions.  People  also  often  use  highly  redundant  evi¬ 
dence  as  If  It  were  completely  Independent  information,  In  effect,  counting  as 
two  sources  what  Is  really  Just  one.  Cues  which  vary  are  given  more  weight  than 
stable  cues,  which  Is  logical,  but  people  sometimes  do  not  notice  when  a  vari¬ 
able  cue  becomes  stable,  and  may  go  on  giving  It  I  ight  weight  on  the  mistaken 
assumption  that  it  continues  to  vary.  The  more  cues  that  are  available,  the 
more  confidence  people  feel  about  their  estimates,  even  though  they  often  fall 
to  Integrate  al  I  the  additional  information  Into  their  Judgments,  or  to  make 
more  accurate  Judgments  with  the  additional  information.  People  seek  out  those 
data  cues  which  are  most  I  Ike  the  answer  that  is  being  sought.  To  estimate  a 
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Figure  6.  Problems  in  Assigning  Weight  to  Data 


o  Lack  of  introspective  accuracy 

o  Reliance  on  redundant  or  highly 
correlated  cues 

o  Emphasis  on  variable  cues 

o  More  cues,  more  confidence 

o  Cue-response  compatibility 

o  Salience  and  vividness 

—  Case  studies  versus  base  rates 

—  Neglect  of  nonevents 


future  attack  submarine  threat,  for  example,  an  anaiyst  might  seek  data  on 
attack  submarines,  although  data  on  naval  aircraftor  space  systems  might  be 
equally  Important  to  answer  the  question.  Highly  salt  ent  and  vivid  data  are 
given  more  weight  than  statistical  data,  or  Information  on  nonevents.  For 
example,  although  analysts  sometimes  use  nonevents  In  their  estimates,  Important 
nonevents  are  often  not  attended  or  analyzed. 

Categorizing  Data 

As  analysts  perceive  and  weigh  data  they  also  categorize  them  —  perception 
Involves  the  act  of  categorizing.  That  Is,  people  attach  a  description  to  a 
piece  of  Information  to  code  It  for  memory  and  later  Inferential  work.  Mental 
theories  and  conceptions  aid  In  this  process  of  classifying  and  organizing 
perceptions  and  facts. 

Perhaps  the  major  difficulty  people  have  In  categorizing  complex  phenomena 
Is  In  trying  to  Impose  etther/or  distinctions  on  objects  or  events  which  have 
too  many  dimensions  and  too  few  cut  and  dried  boundaries  to  be  so  narrowly 
pigeon-holed.  Psychologists  have  found  that  people  typically  do  not  rely  on 
elther/or  categor Izattons  of  complex  objects  or  events  In  making  Judgments  but 
Instead  seem  to  make  Judgments  as  to  whether  the  object  Is  more  or  less  like  a 
prototype  or  schematic  model  of  the  category  In  question.  In  other  words, 
category  boundaries  are  fuzzy  and  probabllstlc  rather  than  hard  and  fast. 

The  fuzzy  nature  of  complex  natural  categories  and  the  difficulty  In 
specifying  ait  the  features  which  give  an  object  or  event  a  family  resemblance 
to  a  category  may  contribute  importantly  to  conflicts  between  Intel! Igence 
analysts.  For  example,  there  are  characteristics  of  Soviet  ships  that  suggest 
they  would  be  effective  deterents  to  I ntervent Ions  during  crises  by  Western 
aircraft  carrier  task  groups.  These  and  other  character!  sties  of  These  ships 
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give  them  capabilities  to  serve  In  a  "b  I  ue  belt  defense  plan"  aga!  nst  nuclear 
strikes  from  the  sea  on  the  Soviet  homeland.  Similarly,  deployments  of  Soviet 
ships  share  features  that  relate  to  both  the  diplomatic  and  strategic  missions. 
Deciding  whether  a  particular  event  was  diplomatic  or  strategic  In  character  can 
be  very  difficult  because  of  the  fuzzy  boundaries  of  these  categories.  Often 
analysts  can  do  no  more  than  to  conclude  that  a  particular  event  was  more  like 
one  category  and  !  ess  I  Ike  the  other  but  a  I  so  a  little  like  both.  Nor  does  1 1 
seem  to  be  any  easier  to  categorize  doctrinal  writings,  military  equipment 
construction,  operational  or  deployment  patterns,  or  diplomatic  events;  all 
these  complex  events  can  be  categorized  as  having  a  family  resemblance  to  stra¬ 
tegic  or  diplomatic  missions,  offensive  or  defensive  objectives,  cautious  or 
expansive  tendencies.  At  best  the  analyst  will  be  able  to  make  only  probablis- 
tlc  Judgments  as  to  the  categories  in  which  an  event  belongs.  Debating  pigeon¬ 
holes  Is  far  less  useful  than  attempting  to  measure  the  strength  of  family 
resemblances. 

It  Is  Important,  when  generalizing  from  a  sample  of  data,  to  be  conscious 
that  certain  data  sampl  I  ng  methods  bl  as  the  I nformatlon  I n  predl  catbl  e  ways. 
One  such  bias  Involves  roles.  For  example,  the  role  of  deterrer  In  a  crisis, 
t.e.,  the  party  that  acts  to  deter  another  party,  has  a  built-in  role  conferred 
bias.  Namely,  It  the  adversary  does  nothing,  the  deterrer  can  credibly  claim 
success  In  deterring,  even  though  the  deterrer  actually  may  have  done  little  and 
the  lack  of  action  by  the  deterred  was  due  to  other  factors  (e.g.,  the  adversary 
lacked  any  Incentive  to  act).  The  deterrer  Is  a  little  Ilk*  the  man  who 
scatters  corn  flakes  around  himself  to  keep  the  Bengal  tigers  away,  when  you 
point  out  that  there  are  no  tigers  In  the  neighborhood,  he  beams  at  you  and 
takes  credit  for  hfs  amazing  deterrent  severs. 

A  second  type  of  sampl  Ing  bias  occurs  when  an  analyst  attends  a  part  hut 


33 


not  ai  I  of  a  data  configuration  (see  Figure  7).  in  panel  A  of  the  hypothetical 
example,  the  entire  data  configuration  is  shown.  Various  biases  can  be  seen  in 
panels  B,  C,  and  0.  When  the  whole  pattern  is  considered  all  one  would  want  to 
conclude  is  that  Soviet  naval  events  of  all  types  are  twice  as  frequent  in  the 
latter  time  period.  Biased  sampl Ing,  however,  might  lead  to  the  conclusions 
that  Soviet  diplomatic  naval  activity  has  doubled,  or  that  strategic  forward 
deployment  has  doubled,  or  that  the  Soviets  are  now  twice  as  likely  to  engage  in 
strategic  as  opposed  to  diplomatic  forward  deployments. 

Real  examples  of  such  biased  sampling  can  be  found  in  naval  analysis.  For 
example,  McGwire  (1976:  166)  argued  that  the  Soviets  surveyed  the  Indian  Ocean 
from  1967  to  1969,  making  many  diplomatic  port  visits  to  scout  out  facilities, 
and  then  visited  very  few  ports  thereaf ter,  having  found  the  facilities  they 
needed.  Peter ten  (in  Dlsmukes  and  McConnell,  1979:  91-925  d'sagreed,  writing: 


While  It  Is  true  that  a  drop  tn  diplomatic  vlstls  to  Indian 
Ocean  ports  was  registered  In  1970,  it  Is  not  true  that 
"very  few"  visits  have  been  made  elsewhere  In  the  region 
since  then.  Between  1970  and  1971,  for  example,  no  fewer 
than  30  diplomatic  visits  were  made  to  Indian  Ocean 
countries  other  than  Somalia  and  South  Yemen.  In 
comparison,  only  23  were  made  to  Med l terranean  ports  during 
the  same  period. 


The  total  data  configuration  is  shown  In  the  figure  (see  Figure  9).  There 
were  nearly  twice  as  many  Indian  Ocean  visits  as  Mediterranean  ones,  but  In  the 
I  a*ter  per  iod  the  ratio  of  visits  Is  not  2  to  1  but  1 .4  to  1 .  Wh  1 1  e  v  I  si  ts  per 
year  Increased  In  the  Mediterranean  over  the  two  periods  from  2.3  to  4,  In  the 
Indian  Ocean  they  decreased  from  10  per  year  to  3.6.  Petersen's  comparison  of 
30  to  28  visits  is  biased  and  ml  srepresenf  s  the  far  higher  level  of  overall 
visiting  in  the  Indian  Ocean  and  the  sharp  decline  of  visiting  in  that  ocean  In 
the  latter  time  period  to  a  level  comparable  to  that  In  the  Medl terranean. 
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Figure  7.  Hypothetical  Example  of  Biased  Sampling 


A.  Soviet  Naval  Events 
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B.  Column  Bias: 


(a)  Diplomatic  Forward  Perspective 
<b)  Strategic  Forward  Perspective 
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C.  Row  Bias:  New  Soviet  Navy  Perspective 
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Figure  3.  Number  (and  Percentage  of  Diplomatic 
Port  Visits  in  the  Indian  Ocean  and  Mediterranean 
by  the  Soviet  Navy  1967-1969,  1970-1976* 


1967-1969 

1970-1976 


Total  1967- 
1976 


*  Data  from 
McConnell 


Indian  Ocean 

Mediterranean 

Total 

30  (43%) 

7  (20%) 

37 

39  (57%) 

28  (30%) 

67 

69  (100%) 

35  (100%) 

104 

Petersen,  Table  3.2  (p.  92  in  Dismukas  and 
1979.) 


data  Covariations 
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Analysts  must  be  able  to  discern  relationships  between  sets  of  data  to 
Infer  that  one  set  of  evorrs  covaries  with  another  set.  For  example,  military 
analysts  must  assess  the  covariation  between  an  adversary's  strategic  needs  and 
military  activities,  between  political  commitments  and  crisis  behavior,  between 
capabilities  and  actions,  etc.,  to  understand  and  predict  military  intentions. 

People  have  difficulty  perceiving  unexpected  covartations  and  tend  to 
perceive  expected  covariations  even  when  there  are  none.  That  is,  people  are 
far  too  ready  to  detect  theory-driven  covariations  in  the  data  they  process  and 
unable  to  see  unexpected  data-driven  covariations  unless  they  are  overwhelmingly 
strong.  Psychologists  have  found  that  people  underestimate  covariations,  I.e., 
strong  correlations  (e.g.,  r  *  .6  to  .8)  are  perceived  as  weak  relations,  and 
near  perfect  covariations  (r  *  .8  to  1.0)  are  perceived  as  merely  strong 
relations.  Unexpected  relationships,  even  when  near  perfect,  tend  to  be  missed. 

Unless  an  analyst  expects  to  find  a  covariation  relationship,  he  or  she 
will  tend  not  to  notice  one  unless  it  is  very  strong.  Consequently,  strong 
relations  may  go  unnoticed,  or  underestimated.  Potential  indicators  of  future 
activity  may  be  overlooked  and  Important  predictable  patterns  may  be  neglected. 

On  the  other  hand,  psychologists  find  that  people  will  perceive  an 
expected,  theory-driven  covariation  even  when  none  exists  in  the  data  set.  What 
seems  to  happen  is  that  positive  occurrences  are  noted,  i.e.,  those  cases  when 
the  two  expected  events  do,  in  fact,  occur  together.  Those  cases  where  one,  but 
not  the  other,  event  occur  are  neglected.  The  analyst  thus  amasses  a  convincing 
list  of  confirmations  of  the  relationship  but  has  Ignored  all  the  di sconf irm ing 
cases. 

Such  difficulties  in  assessing  covariations  can  be  found  In  naval  analysis 
(see  Figure  9).  For  example,  one  analyst  argued  that  Soviet  diplomatic  naval 
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Figure  9.  Problems  in  Assessing  Covariations 


o  Theory-driven  correlations  are  perceived,  even  if  absent 
in  the  data. 

Theory:  The  expansion  of  Soviet  diplomatic 

visits  in  the  Indian  Ocea,  1967-1976. 

Data:  Correlation  between  year  and  number 

of  visits:  -.38. 


o  Data-driven  correlations  are  not  perceived  if  absent  from 
theory. 

Theory:  The  intensity  of  diplomatic  visits 

reflects  the  prominance  of  political 
concerns  in  the  Indian  Ocean  squadron ' s 
mission  structure. 

Data:  Correlation  between  ship  days  and 

diplomatic  visits:  -.65. 

Correlation  between  ship  days  and 
lengths  of  visit:  -.83. 


o  Illusory  Correlations 

e.g. ,  naval  presence  and  coup  occurrences. 


38 


vfslts  (n  the  Indian  Ocean  expanded  In  the  years  1967  to  1976.,  reflecting  Soviet 
Interest  In  that  area.  This  Implies  a  positive  relation;  more  visits  per  year 
as  the  years  go  by.  In  fact  the  rel  atlon  Is  negative:  there  were  less  visits 
as  the  years  went  by  In  this  period.  The  data  suggest  thfare  was  a  contraction, 
not  an  expansion,  of  Soviet  diplomatic  visits  in  the  Indian  Ocean.  On  the  other 
hand,  unexpected  correlations  can  go  unnoted.  One  analyst  expected  that  as  the 
Soviet  Nava!  Squadron  spent  more  ship  days  In  the  Indian  Ocean,  the  promlnance 
of  Soviet  political  concerns  would  intensify  diplomatic  visits.  The  data 
reflected  Just  the  opposite;  the  more  ship  days  the  Soviets  spent  In  fhe  Indian 
Ocean,  the  fewer  the  number  and  the  shorter  the  duration  of  diplomatic  visits. 
These  unexpected  (and  unnoted)  negative  relations  were  quite  strong.  Finally, 
when  a  theory  suggests  a  relationship  should  exist,  coincidental  co-occurrences 
may  be  taken  as  confirming  evidence.  On  several  occasions  the  presence  In  the 
area  of  Soviet  ships  has  been  Interpreted  as  related  to  the  occurrence  of  coups, 
or  crises,  although  there  was  I  Ittle  evidence  to  substantiate  any  connection 
between  the  events  and  the  Soviets  presence. 

Cause  and  Effact 

Covarl  atlons  are  I  mportant  I  n  thel  r  own  right,  but  al  so  because  they  are 
crucial  data  for  Inferring  cause  and  effect  relationships.  The  analysis  of 
causes  Is  one  of  the  Intelligence  analyst’s  major  objectives;  having  causal 
understanding  of  an  adversary  enables  you  to  predict  his  future  actions. 
Determining  causes  Is  one  of  the  most  difficult  of  Intellectual  and  Information 
processing  operations,  however,  and  one  for  which  a  myriad  of  analytic  aids  are 
not  Just  useful  but  often  necessary.  Our  Intuitive,  unaided  reasoning  and 
Judgments  about  cause  and  effect  are  often  In  error.  Even  scientists  sometimes 
find  nonexistent  causes  for  events. 

Psychologists  find  that  there  Is  a  strong  tendency  to  perceive  as  having  a 
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causal  role  anything  on  which  we  focus  our  attention  when  considering  cause  and 
effect  (see  Figure  10).  Futher,  If  an  actor  Is  perceived  to  have  a  certain 
Intention,  and  an  event  occurs  bringing  about  the  desired  outcome,  there  Is  a 
strong  tendency  to  view  the  actor  as  having  caused  the  event,  although  the  event 
may  have  occurred  for  other  reasons.  We  attribute  the  behaviors  of  actors  we 
are  observing  to  their  Intentions,  dispositions,  and  motives  and  underestimate 
the  degree  to  which  their  behavior  is  controlled  by  the  environment,  the  situa¬ 
tion,  or  the  contoxt  they  are  in. 

To  the  degree  an  actor's  behavior  has  Important  consequences  and  the  actor 
can  foresee  the  consequences  of  his  actions,  the  more  we  attribute  the  actor's 
behavior  to  a  profound  motive.  That  Is,  the  cause  Is  perceived  to  match  the 
effect:  if  the  effect  is  profound,  the  motive,  or  cause,  must  also  be  equally 
profound.  This  Is  a  manifestation  of  the  representat I veness  heuristic,  the 
tendency  to  assume  that  features  of  the  cause  shoul  d  be  representative  of  the 
features  of  the  effects.  The  problem  Is  that  rather  trivial  causes  and  motives 
can  lead  to  very  profound  consequences. 

If  an  actor  Is  oei-celved  +o  foresee  his  actions'  consequences,  the  observer 
tends  to  attribute  the  actions  to  Intention.  For  example,  the  Soviet  Union's 
war  fighting  outlook  Implies  foresight;  the  anticipation  of  general  nuclear  war. 
There  Is  a  tendency  to  assume  that.  If  the  Soviets  are  able  to  plan  and  prepare 
for  such  a  war,  they  may  Intend  to  wage  such  a  war  to  w  In  their  own  ends.  The 
Soviets  may  thus  be  seen  as  building  a  nuclear  blackmail  capability.  Of  course. 
It  Is  also  possible  that  the  Soviets  have  no  confidence  In  their  own  deterrent 
capability,  and  prepare  for  nuclear  war,  not  optimistically  for  blackmail,  but 
pesslmlstlcal ly  out  of  a  foar  that  they  cannot  forestall  It. 

In  "magical  thinking",  the  features  of  effects  are  used  to  guide  the  search 
for  causes  —  causes  a-e  sought  which  resemble  effects.  Perhaps  the  worst 
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Figure  10. 


Problems  in  Assessing  Causes  and  Effects 


o  Dispositional  vs.  situational  attributions 
e.g.,  explanations  of  forward  deployment 

o  Profound  motive  fallacy 

o  Foreseeability  implies  causality 

o  Magical  thinking  —  causes  resemble  effects 
e.g.,  capabilities  cause  intentions,  can  do  — 
will  do 

o  Minimal  causation 
o  Causal  hydraulics 

e.g.,  desire  for  military  superiority  precludes 
desire  for  detente 

o  Parsimony  and  " Indiscriminant  Pluralism" 
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example  of  magical  thinking  Is  when  the  analyst  lets  himself  believe  the 
adversary's  Intentions  (causes)  follow  from  or  resemble  their  military 
capabilities  (effects).  In  particular,  there  Is  the  danger  of  calculating  the 
worst  the  adversary  could  do  and  then  reasoning  that  that  Is  what  the  adversary 
will  da  Al ternatively,  the  analyst  may  be  tempted  to  calculate  the  limits  of 
the  adversary's  capability  and  then  estimate  that  the  adversary  would  not 
undertake  missions  that  exceed  those  apparent  limits.  "Minimal  causation" 
refers  to  the  tendency  to  accept  the  first  reasonable  cause  that  fits  the  deta 
as  the  only  cause.  After  peopl e  find  a  pi ausi bl e  cause  to  expl  al n  an  effect, 
they  seem  to  stop  searching  and  the  hy  potties  Is  that  multiple  causes  contribute 
to  an  effect  Is  rarely  tested. 

"Causal  hydraulics*  are  a  colorful  way  of  describing  the  tendency  to 
believe  that  causes  compete  and  compensate  for  each  other  In  producing  effects. 
That  Is,  an  analyst  might  believe  that  If  strong  political  forces  are  producing 
some  behavior,  tnen  military  or  naval  factors  cannot  also  be  contributing  at  the 
same  time.  The  notion  that  bureaucratic  politics  shape  an  adversary's  behavior 
and  that  competing  agencies  and  Interests  are  contending  for  Influence  Is 
especially  subject  to  causal  hydraulic  reasoning.  The  danger  Is  that  necessary 
causes  may  be  mislabeled  as  sufficient  causes. 

A  final  problem  In  causal  reasoning  Is  the  tendency  to  seek  a  plausible 
cause  tor  every  behavior.  This  can  easily  lead  to  what  one  historian  termed 
"Indl  scrim  inant  pluralism*  —  an  attempt  to  propose  a  cause  for  everything.  The 
problem  w  l  th  efforts  to  expl  a  I  n  everythl  ng  Is  that  one  ends  up  with  too  many 
explanations  and  no  ability  to  predict.  There  Is  no  way  of  knowing  which 
explanation  to  use  tor  the  future  situation.  This  Is  why  scientists  seek 
parsimony,  the  fewest  possible  causes  sufficient  to  explain  and  predict.  The 
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smaller  the  number  of  causal  explanations  that  can  account  for  events,  the 
better  the  chance  these  causes  will  predict  the  future. 

Prediction 

Predicting  and  forecasting  are  central  tasks  in  Intelligence  analysis. 
Analysts,  however,  have  been  charged  with  being  unable  to  accurately  predict 
either  the  capabilities  or  Intentions  of  adversaries.  Some  recent  prediction 
failures  are  shown  In  the  figure  (see  Figure  11). 

Many  explanations  of  failures  of  Intelligence  predictions  seek  the  causes 
In  unique  characteristics  of  Intelligence  estimation.  These  explanations  are 
made  I  ess  compel  ling  by  the  fact  tha  forecastl  ng  and  predlctl  ng  are  not  done 
with  much  accuracy  In  other  fields.  Recent  reviews  by  Ascher  (1978)  and  by 
Hogarth  and  Makrldakls  (1979)  of  population,  economics,  energy,  transportation, 
technology,  and  business  forecasts  found  that  prediction  In  these  areas  Is 
rarely  accurate.  This  suggests  that  problems  of  forecasting  and  prediction  are 
general  ones,  not  limited  to  Intelligence  analysis. 

People  need  to  perceive  their  world  as  orderly  and  predictable.  The  more 
people  believe  they  control  events  or  that  events  are  controllable  (even  If  such 
control  Is  Illusory),  the  more  predictable  the  world  seems  to  them.  This  Illu¬ 
sion  cf  control  can  lead  to  predictions  made  with  great  confidence  of  events 
that  are  In  tact  controlled  by  chance.  Forecasters  may  perceive  the  future  as 
more  predictable  then  It  Is  simply  because  of  their  efforts  to  predict  tt. 

People  see  pattern  and  order  where  none  exists  because  of  their  need  for  an 
orderly  world.  There  are  extremely  strong  perceptual  tendencies  to  structure 
the  environment  and  make  sense  out  of  It  In  order  to  organize  perceptions. 
People  rarely  consider  the  possibility  that  the  environment  may  have  random  or 
probabl 1 Istlc  elements.  Instead  they  see  events  as  determined  and  fixed  or 
ordered  by  regular  mechanistic  (not  probabl I Istlc)  processes. 
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Figure  11.  Prediction  Problems 


o  Prediction  Failures 

-  Bombers 

-  ICBMs 

-  iRBMs 

-  Launchers 

-  MIRVs 

-  Defense  Expenditures 

-  Surprise  Attacks 

-  Crises 

-  Peace  Offensives 


o  General  Causes 

-  Illusion  of  Control 

-  Illusion  of  Order 

-  Faulty  Assumptions 

-  Unreliability  of  Judgment  Vice 

Statistical  Relations 

-  Illusions  of  validity  and  overconfidence 

-  Inappropriate  Techniques 

-  Theory-driven  overprediction 


Forecasters  often  reach  different  conclusions  which  all  reflect  a  similar 
biased  Inaccuracy.  Ascher  (1978)  concluded  that  forecasters  making  estimates  In 
the  same  time  period  all  tend  to  share  the  same  key  assumptions,  and  to  the 
degree  these  are  wrong,  tend  to  make  the  same  mistakes.  The  analysts'  key  or 
core  assumptions  may  be  much  less  carefully  scrutinized  than  data  or  methods, 
but  bad  core  assumptions  may  produce  greater  error  than  flawed  data  or 
Inappropriate  methods. 

For  many  medium-  and  short-term  forecasting  tasks,  simple  statistical  trend 
extrapolations  are  often  more  accurate  than  are  analytic  Judgments,  or 
theoretically  elaborate  or  methodologically  elegant  forecasting  methods.  People 
have  very  high  confidence  In  the  superior  accuracy  of  their  Judgments,  a  level 
of  confidence  unwarranted  by  the  demonstrated  Inaccuracy  of  their  predictions 
relative  to  simple  statistical  methods.  People  are  sometimes  most  confident 
when  they  are  least  accurate. 

Often  forecasters  employ  highly  I  nappropr  I  ate  techniques  which  virtually 
assure  an  Inaccurate  prediction.  One  very  overworked  and  abused  technique  Is 
Oelpht,  The  famous  economist,  Paul  Samuel  son,  noted  that: 

the  greatest  error  In  forecasting  Is  not  realizing  how 
Important  are  the  probabilities  of  events  other  than  those 
everyone  Is  agreeing  upon. 

To  the  extent  that  Samuel  son  Is  right  (and  there  are  several  reasons  why  he 
probably  Is)  then  a  technique  like  Oelphl  Is  doomed  to  Inaccurate  predictions 
because  It  focuses  the  experts'  attention  upon  the  probabilities  on  which 
everyone  has  agreed.  Oelphl  systematically  seeks  out  the  extreme  predictions 
and  the  unexpected,  low  probability  conjectures,  leaving  only  the  middle  range, 
concensus  probabilities  most  of  the  experts  had  already  thought  about.  The 
extremes  that  Oelphl  eliminates,  however,  are  the  most  likely  to  predict 
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surprises  and  unexpected  events.  Oelphl  Is  appropriate  only  if  surprises  or 
unexpected  events  are  not  going  to  occur.  But  if  we  can  be  assured  of  that 
degree  of  orderliness,  we  do  not  need  Delphi;  simple  statistical  extrapolations 
will  do  better  and  cost  less. 

ThflflCX 

The  use  o;:  theory,  the  last  analytic  step,  Is  much  to  be  commended  In  any 
prediction  or  forecasting  effort.  There  Is  the  chance,  however,  that  the  theory 
will  suggest  more  order  and  predlctabi  I  I  ty  in  the  data  than  really  exist,  lead¬ 
ing  analysts  to  over-predlct.  That  Is,  the  analyst  with  a  theory  may  make  more 
extreme  predictions  with  greater  confidence  than  the  data  and  the  analyst's 
prediction  record  would  warrant  (see  Figure  12).  One  of  the  main  reasons  for 
using  theories  Is  to  make  accurate  predictions.  The  analyst  must  take  care  not 
to  use  theory  wishfully,  and  to  keep  predicting  distinct  from  theory  test  1  ng. 

Creating  and  modifying  theories  of  the  adversary  should  be  one  of  the  major 
goals  of  the  intelligence  analyst.  Whether  they  acknowledge  them  or  not, 
analysts  are  always  using  something  like  a  theory  as  they  proceed  through  the 
prevtous  stx  estimation  steps.  The  danger  of  using  theory  Is  that  It  can  so 
easily  and  pervasively  color  the  other  data  processing  steps,  leading  to  percep¬ 
tions  of  data,  covariations,  and  cause  and  effect  that  are  not  accurate  or 
val l d. 

Philosophers  of  science  argue  that  the  best  procedure  for  refining  and 
testing  theory  is  to  try  to  disprove  It  In  whole  or  part,  in  general,  however, 
people  (Including  scientists)  tend  to  test  theories  by  the  far  less  productive 
and  less  valid  method  of  accumulating  confirming  evidence.  A  theory  that  sur¬ 
vives  repeated  attacks  t  s  I  ikely  to  be  sore  val  I  d  than  one  that  has  masses  of 
confirming  data  to  support  it  but  has  never  been  subjected  to  a  dlsconf Innation 
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Figure  12. 


Theorizing  Problems 


o  Theory-biased  consideration  of  evidence 
o  Confirmation  bias 

o  Difficulty  of  using  disconf irming  evidence 

o  Infrequency  of  explicit  multiple  hypothesis 
testing 

o  Perseverance  of  discredited  theories 
o  Overconfidence  —  inference  versus  perception 
o  Failure  to  learn  from  experience 


effort.  It  Is  no  simple  matter  to  arrange  valid  dlsconf irmatlon  tests  for 
complex,  real-world  phenomena;  It  Is  difficult  to  establish  the  necessary  con¬ 
trols  and  to  sufficiently  Isolate  the  crucial  elements  of  the  theory  to  deter¬ 
mine  the  meaning  of  the  tests.  Similar  problems  occur  with  confirmation  tests 
but  they  are  psychologically  more  easily  overlooked.  Analysts  tend  mreh  more  to 
amass  confirming  data  for  a  theory  than  to  explicitly  attempt  to  dlsconf Irm  a 
theory. 

Like  most  people,  analysts  Involved  In  hypothesis  testing  rarely  set  out  to 
pit  opposing  theories  against  each  other  and  explicitly  test  them  against  the 
same  data  bases.  Some  efforts  are  very  worthwhile  from  a  methodological 
v  l  ewpol nt,  but  requl re  dill  gent  and  r I gorous  methods  to  keep  or der ly  all  the 
logical  imp  I  l  cat  Ions  of  the  data.  It  Is  also  difficult  to  eval  uate  cases  of 
partial  data  support,  l.e.,  there  Is  no  good  metric  for  measuring  partial 
support  for  a  theory. 

Psychologists  and  historians  of  science  have  found  that  people  In  general 
and  scientists  In  particular  are  often  unwilling  to  give  up  their  theories  even 
In  the  face  of  massive  discrediting  evidence.  Once  a  theory  Is  accepted  It 
becomes  quite  easy  to  explain  away  any  conflicting  data  as  artlfactual,  biased. 
Improperly  analyzed  or  Interpreted,  Irrelevant,  etc.  An  accepted  theory  quickly 
becomes  entwined  with  other  theories  and  beliefs  which  tend  to  support  and 
bolster  the  theory  when  conflicting  or  anomalous  data  are  received.  A  far 
higher  standard  of  evidence  Is  demanded  for  dlsconf Irmlng  evidence  than  Is  re¬ 
quired  for  confirming  evidence.  Such  an  attitude  Is  approriate  only  to  the 
degree  that  the  theory  is  making  proven,  accurate  predictions.  Analysts  should 
be  mere  skeptical  If  the  theory  Is  unproven  In  application,  or  has  a  checkered 
pest  In  making  predictions  and  forecasts. 
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People  have  remarkably  high  levels  of  confidence  In  their  own  analytic  and 
Inferential  powers  —  actually  overconf I dence,  since  they  are  often  more  confi¬ 
dent  In  their  Judgments  than  they  are  accurate.  On  the  other  hand,  people  also 
seem  reluctant  to  credit  as  accurate  their  perceptual  senses  and  memory  for 
psychophysical  Judgments.  People  can  make  very  accurate  Judgments  with  these 
capabilities  but  seem  to  have  iess  confidence  In  them  than  In  their  deductive 
and  Inferential  reasoning  powers.  This  suggests  that  analysts  might  try  to  make 
greater  use  of  methods  that  capital  Ize  on  their  perceptual  sense  (e.g.,  using 
data  reduction  and  display  techniques  that  employ  graphic  comparisons)  and  rely 
on  mental  Inference  and  deduction  only  to  the  degree  that  some  external  criteria 
of  validity  can  be  summoned  for  assistance. 

The  characteristics  reviewed  above  of  how  people  use  and  maintain  theories 
make  It  very  hard  for  people  to  learn  from  experience  that  their  theories  are 
wrong  or  Inaccurate.  Sad  theories  can  survive  a  great  amount  of  painful 
experience.  People  tend  to  assume  a  deterministic,  well-ordered  world  with  neat 
conceptual  boundaries  and  they  have  difficulty  using  the  stochastic  data  the 
world  usually  generates. 

Fortunately,  many  of  these  same  Information  processing  problems  have  been 
encountered  before  In  different  fields,  and  methods  have  evolved  to  cope  with 
them.  Some  of  these  aids  and  their  features  are  described  In  the  next  section. 

C.  ANALYTICAL  METHODS 

For  the  purpose  of  this  research  project  we  have  considered  four  general 
categories  of  analytical  aids  to  the  problem  of  estimation  of  Intentions. 

I.  Judgmental  methods  employing  sagacltv  understand! ng  of  an 
analyst  or  estimator  of  relationships  between  subtle  cues  and 
behavior),  control  (Intelligence  experiments  that  provide 
predictive  Information  by  controlling  the  Information  available 
to  an  adversary),  and  acumen  (the  result  of  pure  reasoning  and 
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the  appreciation  of  the  others*  point  of  view  In  a  par*Icular 
place  and  time). 

2.  Analytical  aids  Involving  such  techniques  as  event  coding  and 
content  analysis  that  support  specific  elements  of  the  estimation 
process. 

3.  Extrapolation  models  based  on  past  events  and  data,  and 

4.  Structural  represen+at I ons  or  models  which  take  Into  account 
structural  realities  or  changes  with  respect  to  leadership, 
capab 1 1 1  ties,  and  other  state-of-the-wor I d  factors. 

These  categories  are  not  mutually  exclusive  In  that  specific  analytical  aids  may 
fal  I  Into  several  of  the  categories.  Further,  any  estimate  of  Intention  must 
generally  draw  upon  all  four  categories  of  aids.  Clearly,  the  ultimate 
estimation  must  be  based  on  Judgments  of  the  reality  of  critical  assumptions, 
situations,  and  responses  that  might  be  Indicated  by  an  analytical  aid.  Quanti¬ 
tative  analysis  In  the  extrapolation  or  structural  representation  categories  can 
comprise  between  10  and,  say  Q0%  of  an  Intention  estimation  depending  on  the 
situation,  in  ai I  cases  subjective  Judgments  must  be  aopl led  at  some  stage. 
Also,  some  portion  of  most  estimation  problems  will  be  amenable  to  quantitative 
analysis.  The  art  of  estimation  is  to  select  and  blend  appropriate  methods  and 
the  objective  of  this  research  Is  to  assist  In  that  process. 

Specific  analytical  methods  that  may  be  used  In  supprot  of  Judgmental 
analyses  Include: 

o  scenario  writing  to  explore  a  range  of  cl rcumstances  and 
responses  In  an  Internally  consistent  framework  that  can  be 
reviewed  and  evaluated  by  a  third  party 

o  availability  and  representat I ve  heuristics. 

The  heuristics  can  be  aids  or  mental  traps  depending  on  the  situation  and  their 
application,  but  must  be  listed  since  they  represent  paslc  psychological  proces¬ 
ses  In  the  perception  and  integration  of  data.  The  availability  heuristic  deals 
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with  the  ease  with  which  memories  and  stereotyped  constructions  of  events  can  be 
retrieved  by  an  individual.  The  representative  heuristic  in  a  judgment  strategy 
that  involves  the  use  of  salient  features  of  a  sample  to  infer  its  membership  in 
a  specific  category  or  class  represent! ng  behavior  or  response. 

Analytical  aids  to  the  extrapolation  category  of  estimation  include  the 
statistical  methods  of  covariant  analysis  and  regression  analysis.  These 
techniques  may  be  used  to  extrapolate  both  intentions  and  capabilities  based  on 
past  observations.  Both  probab 1 1 i st 1 c  and  determ i n i n ! stl c  methods  may  be 
employed. 

A  wide  range  of  analytical  aids  may  be  drawn  for  the  structural 
representations  category  of  estimation.  This  category  of  analysis  breaks  free 
from  the  problems  inherent  In  the  extrapolation  processes,  but  requires  a  more 
detailed  understanding  of  cause-effect  rel  ationships  and  character  ization  of 
changes  in  capabilities;  leadership,  and  other  pertinent  factors.  Techniques 
employed  here  Include  Bayesian  analysis,  factor  analysis,  decision  analysis,  and 
other  structural  modeling  techniques.  Intention  estimates  using  these  methods 
must  draw  upon  rel ated  studies  of  elite  structures,  technical  capab 1 1  1 1! es, 
logistics,  and  deployment. 
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COMPARISON  AND  EVALUATION  OF  ANALYTICAL  AIDS 


This  chapter  describes  a  variety  of  quantitative  aids  that  have  promise  in 
dealing  with  the  analytic  problems  noted  for  the  various  steps  in  the  estimation 
process  In  the  previous  chapter.  It  also  outlines  how  the  Important  analytic 
features  of  quantitative  methods  can  be  assessed  by  the  analyst  considering 
their  use. 

In  describing  these  techniques  we  attempt  to  touch  on  several  aspects.  The 
technique  Is  briefly  described,  if  possible  using  an  example  from  Intelligence 
analysis  (several  cases  are  taken  from  an  earlier  study  of  Sovtet  naval 
analysis,  Stech,  1981).  The  major  theoretical  and  methodological  assumptions  of 
the  method  are  described  and  the  nature  of  the  Input  data  required  ts  outlined. 
The  output  of  the  method  is  discussed  and  related  to  estimation  problems  that 
might  be  alleviated.  Some  other  aspects  that  are  touched  upon  include  the 
descriptive.  Inferential,  and  deductive  power  that  the  technique  might  provide, 
the  startup  and  maintenance  costs,  the  flexibility  of  the  technique  to  deal  with 
new  or  different  research  questions,  the  need  for  special  equipment  training  or 
expertise,  any  any  Interesting  character i st I cs  of  the  methods  (see  Brownel I, 
Stoll,  and  Thomann,  1980  for  a  similar  evaluative  format  for  estimative  and 
analytic  techniques). 

There  are  many  promising  analytic  methods  and  experimental  techniques  that 
apply  to  the  estimative  problem  areas.  Some  of  these  techniques  are  used  now  by 
analysts  (but  apparently  infrequently)  and  have  been  applied  to  various  types  of 
Intelligence  estimation.  Some  have  not  yet  been  used  by  Intelligence  analysts, 
so  far  as  we  know.  All  of  these  methods  are  aimed  at  compensating  for  or 
preventing  the  information  processing  biases  that  can  occur  in  unaided  analysis. 
They  are  all  analytic  aids,  not  substitutes  for  anaivsts  or  analytic  reasoning. 
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The  editor  of  a  recent  compendium  on  quantitative  approaches  to  political 


intelligence  in  the  CIA  observed  (Heuer,  1978:  1): 


The  behavioral  revolution  in  academic  political  science  has 
been  virtually  ignored  by  the  (CIA)  and  the  intelligence 
community  as  a  whole. 


Although  Heuer  saw  the  narrative  essay  as  continuing  to  be  the  dominant  form  for 
intelligence  estimates,  he  recommended  greater  use  of  quantitative  techniques 
that: 

...help  to  trace  the  logical  consequences  of  subjective 
Judgments,  extend  the  mental  capacity  of  the  individual 
analyst,  force  the  analyst  to  make  hts  assumptions  explicit, 
or  help  organize  complexity  (p.  8). 

The  aids  described  below  are  designed  to  enhance  analysis  of  intentions  by 
moving  toward  the  goals  Heuer  listed. 


A.  PERCEIVING  CATA 

The  three  main  problems  regarding  the  perception  of  data  are  (1)  the 
nonpercept  Ion  of  nonevents  and  negative  evi  dence,  (2)  the  use  of  availability 
Information,  and  (3)  the  use  of  representative  features  of  samples  to  estimate 
population  characteristics,  to  the  neglect  of  base  rate  data. 

Ejcaats  anOanayaals 

Analysts  must  render  a  continuous,  undiff erent ! ated  stream  of  information 
on  the  adversary  into  discrete,  discriminate,  descrlbable  entitles.  Perception 
Is  discrete  rather  than  continuous.  The  adversary  is  perceived  as  performing  a 
series  of  discrete  actions.  These  actions  divide  the  stream  of  Information  Into 
segments  or  units. 
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Psychologists  have  begun  to  Investigate  how  people  segment  this  stream  of 
Information  Into  series  of  discrete  action  events.  One  main  finding  Is  that  the 
density  of  events  becomes  greater  the  more  unexpected  the  action.  That  is,  when 
people  observe  highly  organized,  predictable,  stop-by-step  actions,  with  a  clear 
hierarchy  of  subordinate  and  superordinate  goals,  they  tend  to  segment  the 
action  perceptual ly  Into  grosser  units.  People  resort  to  shorter  units  when 
perceiving  unexpected  action.  Organization  of  the  perceived  action  becomes 
extremely  fine-grained  Immediately  after  an  unpredicted,  significant  event 
occurs. 

Most  Intelligence  analysts  are  familiar  with  the  Impact  of  crises  or 
"surprises"  on  normal  operations  —  requirements  on  Intel l Igence  production 
Increase  greatly.  Much  more  fine-grained  analysis  Is  called  for  than  normally. 

One  consequence  of  this  difference  In  event  perception  Is  that  different 
coding  categories  are  appl led  to  the  cases  of  expected  and  unexpected  events. 
Actions  during  a  crisis  may  appear  different  from  everyday  actions  because  the 
former  are  subjected  to  fine-grained  coding,  while  the  latter  fit  grosser,  more 
familiar  schemas.  This  suggests  that  analysts  make  strong  efforts  to  keep  their 
fine-grained  segmentations  comparable  (through  aggregation)  to  their  day-to-day 
coding  of  act  Ions.  This  will  allow  for  meaningful  com  par  I  sons  of  crisis  and 
noncrlsts  episodes.  Such  comparisons  may  be  precluded  If  the  analysts'  fine- 
grain  categories  of  codtng  events  In  crisis  are  not  comparable  with  the  grosser 
normal  coding. 

It  Is  also  Important  that  psychologists  have  found  that  as  perceived 
behavior  becomes  more  motivationally  Important  to  the  percelver,  grosser  coding 
units  tend  to  be  used.  That  Is,  during  an  Important  crisis,  the  consumers  of 
estimates  may  tend  to  use  grosser  codings  of  events  at  the  same  time  that 
analysts  are  using  more  fine-grained  codings.  (This  assumes  that  declslon- 
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makers  and  operators  are  under  greater  motivational  pressures  than  are 
analysts.)  That  Is,  arousal  leads  to  a  tendency  to  focus  on  a  few  relevant  cues 
and  decreases  the  range  of  attention,  while  amplifying  Its  Intensity. 

Several  studies  have  shown  that  people  segmenting  action  Into  larger  units 
make  noutral  attributions  as  to  causes:  I.e.,  the  actions  are  attributed  to 
both  situational  factors  and  dispositions  of  the  actor.  When  using  smal ler 
units  of  analysis,  people  tend  to  attribute  action  to  the  actors  alone,  not  to 
the  situation.  This  suggests  that  analysts  during  crisis  should  be  alert  to  (1) 
a  tendency  toward  "hypervlg! I ance",  or  the  close  monitoring  of  only  a  few 
Indicators,  (2)  the  posslbl  Ity  that  estimate  consumers  may  be  looking  for  very 
gross  organizations  of  action  while  analysts  are  generating  extremely  fine 
analyses,  and  (3)  analytic  categories  for  crisis  action  may  be  Incompatible  with 
the  categories  used  for  normal  actions,  possibly  producing  a  false  analytic 
dichotomy,  especially  regarding  the  causes  of  action. 

A  method  which  might  help  analysts  to  code  a  stream  of  Information  Into 
useful  categories  was  demonstrated  by  O'Leary  and  Coplln  (1575;  182  ff.)  for 
Statement  Department  Intelligence  analysts  using  data  on  conflict  acts  between 
Egypt  and  Israel.  Rules  tor  coding  each  type  of  event  on  a  scale  of  violence 
were  jsed  to  score  the  entire  event  series.  Analysts  were  then  able  to 
graphically  follow  events  In  terms  of  either  the  frequency  of  events  of 
different  levels  of  violence,  or  In  terms  of  the  level  of  violence  Itself.  Such 
graphs  and  codings  enable  the  analyst  to  capture,  respectively,  the  grosser 
rel atlonsh I ps  between  events  and  the  f I ner-gra I ned  details  of  actions.  Sy 
allowing  the  analyst  to  demonstrate  patterns  and  trends  In  events,  coding  rules 
and  methods  aid  In  the  observation  of  nonevents  and  negative  evidence. 
Interruptions,  omissions,  and  nonoccurrences  are  easier  to  detect  against  an 
orderly  background  of  action  trends. 
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Data  on  Frequencies 


The  availability  heurl Stic  Is  fundamentally  the  tendency  to  substitute 
memcrab 1 1 1 ty  for  an  estimate  of  frequency.  To  the  degree  that  more  objective 
data  on  frequency  are  readily  accessible,  the  analyst  should  feel  less  compelled 
to  use  availability  as  a  substitute.  Two  techniques  for  systematic  frequency 
recording  In  Intelligence  analysis  are  event  coding  and  content  analysis. 

Event  coding  has  become  a  staple  method  In  efforts  at  crisis  forecasting 
and  prediction  (see,  e.g„  the  March  1977  Issue  of  International  Studies  Quar¬ 
terly  on  "International  Crisis:  Progress  and  Prospects  for  Applied  Forecasting 
and  Management"  or  Kaplan,  1981  for  a  coding  of  the  political  use  of  Soviet 
military  power).  In  general,  elaborate  rules  for  coding  events  or  actions  are 
specified  which  are  then  applied  uniformly  to  the  stream  of  events.  This  allows 
the  analyst  to  make  quantitative  comparisons  on  any  of  the  dimensions  created  by 
the  coding  scheme.  Given  the  widespread  use  of  event  analysis  In  early  warning 
Intelligence  and  crisis  forecasting.  Its  absence  In  other  areas  of  analysis  Is 
striking.  Some  naval  analysts  make  use  of  event  tabulations  (see  Petersen,  In 
Olsmukes  and  McConnell,  1979:  ch.2,  Tables  2.12-2.15,  for  a  detailed  listing  of 
Soviet  naval  operations),  but  the  events  are  not  elaborately  coded  and  frequency 
analyses  are  uncommon.  Naval  analysts  now  use  some  event  categories  (e.g., 
frequencies  of  diplomatic  port  visits)  In  their  studies,  so  they  do  not  seem 
adverse  to  the  concept.  It  may  be  that  more  complex  coding  of  Soviet  naval 
events  waits  on  a  taxonomy  of  naval  actions  to  provide  the  coding  framework. 
McConnell's  analysis  of  the  "rules  of  the  game"  (ch.  7  In  Olsmukes  and 
McConnell,  1979)  might  provide  an  Initial  step  for  developing  such  an  event 
coding  system,  and  his  classification  of  cases  (Tables  7.1  and  7.2)  provides  a 
rough  coding  scheme  (see  Cohen.  1980.  for  an  assessment  of  "rules  of  the  game" 
analysis  In  estimation). 
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Event  coding  techniques  make  several  methodological  assumptions  the  analyst 
should  consider.  In  order  to  code  events  the  coder  must  receive  some  report  of 
the  event,  a  cable  from  the  embassy,  an  Intel  I  I gence  report,  a  wire  service 
story,  a  news  article.  Those  Individuals  and  agencies  doing  the  reporting  are 
also  affected  by  the  same  event  perception  phenomena  we  outl Ined  above.  As 
events  become  more  unexpected,  the  density  of  reporting  will  Increase.  This 
changes  the  character  of  the  reporting  from  crisis  to  non-crisis  periods,  maktng 
comparisons  across  periods  problematic  unless  specific  efforts  are  taken  to 
control  for  the  finer  grained  perceptions  of  the  crisis  periods.  There  Is 
likely  to  be  a  significant  feedback  effect  from  the  analytic  community  to  the 
policy  community  to  the  reporting  sources.  If  an  embassy,  intelligence  collec¬ 
tion  organizarion,  or  news  gathering  team  learns  that  the  analytic  and  policy 
communities  are  monitoring  a  particular  problem  or  area  they  are  likely  to 
Intensify  their  reporting  of  events  and  will  report  events  that  might  otherwise 
have  been  Ignored.  The  consequence  is  that  the  basic  rate  and  character  of  the 
reports  changed.  This  Is  critical  because  volume  of  reporting  Is  one  Indicator 
used  In  event  analysis.  Positive  feedback  effects  may  significantly  distort 
such  an  Indicator. 

Eve;vf  coding  requires  varying  levels  of  Input  data,  depending  on  the  pur¬ 
pose  of  the  analyst.  If  used  to  monitor  all  world  activity  tn  the  hope  of 
anticipating  crises  and  "hot  spots"  a  continuous  stream  of  events  must  be  coded, 
l.e.,  from  the  public  press  or  monitored  news  broadcasts.  On  the  other  hand. 
Individual  analysts  might  develop  a  coding  scheme  to  track  various  events  over 
time  tor  a  particular  country,  e.g..  Incidents  of  civil  disorder,  requiring 
considerably  less  coding  volume  and  effort.  Like  any  pre-determ  I  ned  coding 
scheme,  event  analysis  Is  vulnerable  to  the  problem  that  the  Initially  estab¬ 
lished  coding  categories  may  be  Inappropriate  for  later  analytic  tasks.  Coding 
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events  In  terms  of  cooperation  and  conflict,  for  example,  might  be  Inappropriate 
If  the  analyst  later  must  determine  variation  in  types  of  conflict.  No  coding 
scheme  Is  sufficiently  flexible-  to  anticipate  all  the  future  research  questions 
the  analysts  may  need  to  answer. 

Content  analysis  has  been  applied  to  Intelligence  problems  since  World  War 
II  (e»g.,  George,  1959)  to  determine  objective  frequencies  for  actions,  events, 
or  statements.  Content  analysis  Is  generally  applied  to  verbal  or  written 
statements  (e.g.,  propaganda,  speeches,  memoirs)  to  determine  such  things  as 
authenticity,  trends  In  semantics  or  rhetoric,  shifts  In  Interests. 

Soviet  naval  analysts  Frledhelm  and  Hehn  (1977)  made  Imaginative  use  of 
content  analytic  methods  to  determine  Soviet  positions  In  Law  of  the  Sea  (LOS) 
negotiations.  They  described  (p.  345)  their  technique  as  follows: 

Soviet  positions  on  f  tve  of  the  major  Issues  that  the  USSR 
had  entered  Into  the  UN  record  were  measured  by  thematic 
content  analysis  of  statements  by  official  speakers  who 
expressed  for  their  governments  a  preferred  pos I t!on...Th t s 
provides  a  systematic  record  of  a  1 1  major  points  made  by  a  1 1 
states  In  these  negotiations  since  they  began  In  1967. 

This  technique  al lowed  Frtedhelm  and  Hehn  to  conclude,  for  example,  that  the 
Soviets  have  been  somewhat  flexible  on  the  issue  of  free  transits  of  straits, 
but  Inflexible  on  fishing  rights.  They  were  also  jble  to  "score1*  national 
positions  on  LOS  Issues  so  that  Soviet  views  can  be  compared  with  U.S., 
Japanese,  or  other  national  positions.  They  also  compared  the  USSR  positions 
with  those  of  Important  Individuals,  e.g..  Admiral  Gorshkov. 

Content  analysis  has  been  used  to  address  tradttlonal  Krem I  I nol ogl ca I 
I  ssues,  e.g.,  what  the  attitudes  of  Sov  let  ell  tes  were  toward  Leon  I  b  Brezhnev 
(Heuer,  1978).  Heuer  analyzed  how  sixteen  Soviet  elites  referred  to  Brezhnev. 
He  found  an  index  of  personal  reference  rank-ordered  the  sixteen  elites  In  terms 
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of  their  political  support  for  Brezhnev  much  like  a  panel  of  three  CIA  experts. 
A  recent  study  by  Kirk  (1980)  of  ai  !  publ  tc  speeches  by  members  of  the  Soviet 
Pol itburo  between  1972  and  1979  offers  some  Interesting  content  *nalys! s  re¬ 
sults.  Ourtng  World  War  II  analysts  became  very  adept  at  Inferring  Nazi  inten¬ 
tions  from  the  analysts  of  the  content  of  Axis  propaganda  (See  George,  1959). 

Content  analysis  has  the  advantage  of  being  systematic  and  having  clear 
rules  of  coding  and  Inference,  aspects  lacking  In  more  impressionistic  modes  of 
Inquiry.  It  has  a  variety  of  drawbacks  however.  It  Is  extremely  labor 
Intensive.  The  reading  and  codtng  of  every  document  or  speech  possibly  relevant 
to  a  particular  tssue  can  require  large  numbers  of  man-hours  and  multiple 
coders,  introducing  the  problems  of  intra-  and  Inter-coder  reliability.  Texts 
In  foreign  language  may  not  code  equivalently  In  translated  form,  requiring 
original  language  coding,  or  validation  experiments  with  parallel  codtng  In  the 
original  language  and  tn  English  to  determine  If  coding  of  the  translations  Is 
feasible. 

Often  the  coding  tn  content  analysis  consists  of  highly  oojectlve  cate¬ 
gories  such  as  frequencies:  how  often  has  a  given  expression  occurred  In  var¬ 
ious  media.  Other  codings  are  more  subjective,  as  when  pronouncements  are  coded 
as  cooperative  or  confllctual.  The  rich  symbolism  In  the  use  of  language  makes 
clear-cut  coding  of  all  linguistic  meaning  Impossible.  For  example,  Irony  Is 
always  subject  to  possible  m  i  si  nterpretatlon,  and  may  not  be  appropr I  ate  I y 
coded.  Euphemism,  tone,  analogy,  simile,  and  Aesopian  language  are  various 
forms  of  soph  I stocated  expression  that  may  confuse  content  coding  and  lead  to 
m  I  scategor  I  zat  I  on  of  expressions.  The  context  of  the  communication  may  or  may 
not  affect  Its  content.  Speakers  may  or  may  not  express  what  is  on  their  minds. 
These  problems  influence  the  I nterpretet l on  of  the  results  of  content  analysis. 
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Each  content  analysis  project  requires  the  establishment  of  coding 
categories  and  rules  for  classification  so  that  the  coders  know  how  to  assign 
content  to  classes.  While  It  Is  these  rules  that  lend  objectivity  to  the 
practice  of  content  analysis,  it  aiso  leaves  the  technique  inflexible  if  new 
research  questions  arise  that  were  not  considered  when  the  original  categories 
were  determined.  New  questions  may  require  the  re-codlng  of  the  same  material 
to  check  for  the  new  categories.  The  most  modern  methods  of  computer  text 
reading  and  text  storage  will  alleviate  this  problem  of  Inflexibility  to  some 
extent,  but  such  techniques  have  not  been  applied  widely  to  content  analysts  and 
It  will  be  some  time  before  a  fully  flexible  machine- assisted  content  analytic 
procedure  exists. 

Features  of  Samples  and  Populations 

The  representativeness  heuristic  Is  the  tendency  (I)  to  assume  that  a 
sample  Is  entirely  representat I ve  of  the  population  from  which  the  sample  was 
selected,  and  (2)  to  neglect  features  of  the  population  that  are  not  In  the 
sample.  The  methods  that  reduce  representativeness  are  those  that  Improve  the 
chances  that  the  analyst's  appraisal  of  a  sample  will  accurately  ref  I ect  the 
features  of  the  population. 

A  common  manifestation  of  the  representat I veness  heuristic  Is  the  tendency 
to  overemphasize  case-specific  Information  and  underemphasize  base-rate  data. 
This  suggests  that  anaysts  should  give  more  attention  to  statistics  on  central 
tendencies  (means,  medians,  and  modes)  and  dispersion  (variance)  In  samples. 
Analysts  rarely  report  such  ! nformatlon  and  typically  seem  no*  to  use  simple 
descriptive  statistics.  Even  the  moving  average  (a  smoothing  statistic), 
commonplace  In  trend  analysis  and  forecasting.  Is  rare  In  Intelligence  ana  lysis 
estimates,  with  the  exception  of  those  that  deal  with  economic  statistics. 
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One  technique  used  by  analysts  to  develop  the  features  of  a  sample  Is 
expert  opinion.  A  panel  of  experts  Is  requested  to  specify  features  or  aspects 
of  a  problem  that  are  Important  for  analysis.  The  analyst  then  collects  data  on 
these  features  for  a  sample  and  uses  these  for  projection  to  a  population.  For 
example,  Ivanoff  and  Murphy  (NPSP.  ch.  7)  developed  nine  composite  technical 
parameters  tnat  could  be  used  ho  assess  and  project  Soviet  technical  progress  In 
antt-shlp  cruise  missiles. 

The  trends  in  these  composite  parameters  are  estimated  and  then  Ivanoff  and 
Murphy  wrote  (p.  153); 


...conclusions  are  drawn  on  future  adversary  systems  that 
will  be  devel  oped...Th  I  s...requ  Ires  a  svnthes  1 2ar  rather 
than  an  analyst...al  I  considerations  of  Soviet  practices  are 
merged  with  the  factual  evidence  of  the  quantitative 
analysis.  Future  systems  are  synthesized  and  described... 


Thorpe  (NPSP.  ch.  8)  used  a  panel  of  experts  to  determine  the  mission  priorities 
for  each  Soviet  ship,  aircraft,  and  submarine.  Since  many  Soviet  naval 
platforms  are  deemed  multi-purpose,  Thorpe's  objective  was  to  quantify  their 
multiple  features. 

Dawes  (1974)  proposes  that  the  role  of  the  expert  In  predictive  systems 
should  be  to  determine  which  variables  seem  to  be  Important  and  how  they  should 
relate  to  the  prediction: 


There  Is  no  way  of  knowing  apart  from  (the  expert)  what 
variables  should  be  looked  at.  And  the  man  knows  what 
variables  to  look  at  only  because  he  knows  something  about 
how  they  predict  (p.  524). 


Once  the  possible  predictive  features  are  Identified,  Dawes  recommends  that  they 
be  systematical  ly  tested  In  a  model  to  verity  that  ths  evre-'t-selacted  variables 
do  predict.  That  Is,  In  contrast  to  the  traditional  use  of  experts,  as  in  the 
Ivanoff  and  Murphy  study,  where  experts  selected  variables,  and  then  synthesized 


Judgements  from  them,  Oaves  recommends  the  experts  select  the  variables  and  the 
synthesis  ana  data  Integration  be  done  mathematically  or  mechanically.  This 
recommendation  Is  based  on  the  fact  that  humans  are  consistently  Inferior  to 
mechanical  Information  Integration  systems  when  synthesizing  complex  data 
patterns. 

The  use  of  experts  to  single  out  predictive  variables  and  the  combination 
of  the  variables  In  a  mathematical  model  has  been  termed  "paramorph I c  represen¬ 
tations"  (Hoffman,  1960)  of  the  experts,  and  the  use  of  the  mathematical  com¬ 
bination  of  variables  to  Improve  the  Judgements  of  the  experts  has  been  termed 
"bootstrapp I ng"  (Oaves,  1971;  Goldberg,  1970).  Models  of  the  exper*  Judges  out¬ 
perform  the  Judges  themselves  because  the  mathematical  Integration  of  Informa¬ 
tion  Is  more  reliable  and  consistent  than  human  Integration.  Human  assessments 
suffer  from  Inconsistencies  and  distortions  that  are  Irrelevant  to  the  problem 
being  analyzed,  whereas  the  mathematical  analysis  is  more  reliable.  The  pre¬ 
diction  model  Is  Immune  from  boredom,  fatigue,  distractions,  or  variable  appli¬ 
cation  of  the  Integration  rules.  To  the  degree  that  the  human  expert  has  less 
than  perfect  reliability,  error  Is  added  to  his  or  her  predictions  and  can  only 
reduce  the  accuracy  of  the  estimate. 

Paramorphlc  and  bootstrap? I ng  models  usually  are  simple  linear  combinations 
of  predictive  variables.  Experts  often  reject  such  simple  models  as  being 
Inappropriate  for  what  they  perceive  as  complex,  nonl  Inear.  conf  Igural  tasks 
(e.g..  SI  ovlc,  1969).  The  evidence  from  repeated  studies  suggests  however  that 
the  experts  perceptions  of  their  tasks  do  not  conform  to  their  own  predictions, 
which  tend  to  be  linear  combinations  (see  Chan,  1961  for  a  review  of  studies). 
Simple  linear  extrapolations  of  known  trends  are  usually  more  accurate 
forecasting  methods  than  complex  prediction  systems.  at  least  for  the  short  and 
medium  term  (Hogarth  and  Makrldakls,  1979). 
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A  psychological  technique  that  has  had  very  little  application  In 
Intelligence  could  aid  great iy  In  systematizing  expert  opinions  on  the  features 
of  events  or  any  other  samples  of  stimuli.  This  Is  the  technology  of  metric  and 
nonmetrlc  multidimensional  scaling,  tree-fitting  and  clustering.  These  methods 
will  be  discussed  more  fully  below,  under  Character  I z  I ng  Data,  but  are 
Introduced  here  to  demonstrate  how  they  may  aid  In  constructing  meaningful 
perceptual  categories  for  analysts  from  expert  or  analyst  generated  data. 

Shepard  (1980)  recently  summarized  research  on  these  methods,  which  are 
"computer  based  methods  for  constructing  representations  of  the  psychological 
structure  of  a  set  of  stimuli  on  the  basis  of  pairwise  measures  of  similarity  or 
confusabll  tty".  These  techniques  yield  three  complementary  representations  of 
psychological  structures:  dimensional  scales,  taxonomotrlc  tree-structures,  and 
cl usterlngs. 

Generally,  this  technique  us-s  Judges  to  assess  the  similarity  between 
pairs  of  stimuli  or  to  sort  stimuli  Into  categories.  A I ternat I ve I y,  stlmul  I 
pairs  can  be  presented  to  people  for  Judgments  of  "same"  or  "different".  The 
similarity  ratings.  In  the  first  case,  or  confusion  scores.  In  the  second,  can 
then  be  mathematical ly  fitted  Into  a  dimensional  soace  or  Into  sets  which  pre¬ 
serve  the  psychological  similarities  and  difference  h*t<.«on  the  stimuli  Items. 
Either  Individual  experts,  or  groups  of  experts,  can  nave  their  Judgments  thus 
scaled  or  clustered.  The  output  of  such  techniques  Is  a  set  of  dimensions  or 
categories  on  which  sttaull  objects  can  be  measured  or  compared. 

In  other  words,  one  means  by  which  Thorpe  (NPSP.  ch,  3)  might  have 
approached  the  problem  of  mission  priorities  of  Soviet  navy  platforms  would  be 
to  present  each  pair  (of  alt  possible  pairings  of  platforms)  and  ask  for  a 
rating  of  the  similarity  of  missions.  These  expert  generated  data  would  yield  a 
set  of  dimensions  or  a  taxonooetrlc  tree  structure  on  which  platforms  with 
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missions  perceived  to  be  similar  would  be  closely  located,  while  platforms  which 
shared  no  missions  (In  the  experts’  opinions)  would  be  found  far  apart.  These 
dimensions  or  the  shape  of  the  taxonomy  tree  would  be  the  fundamental  mission 
aspects  of  the  platforms  In  the  perceptions  of  these  experts.  However,  It  Is 
likely  that  the  clustering  or  a  hierarchical  tree-figure  would  better  represent 
the  mission  variables  perceived  by  the  experts  than  a  set  of  dimensional  scales. 

Why  should  one  use  scaling/clustering  procedures  to  abstract 
dimensions/structures  of  mission  priority  for  Soviet  Naval  platforms?  Why  not 
merely  ask  experts  aoout  these  missions,  as  Thorpe  did?  First,  experts  do  not 
all  perceive  stimuli  In  exactly  the  same  manner.  While  Thorpe’s  method  may 
allow  an  estimate  to  be  made  of  the  differences  between  experts  (e.g..  range  of 
opinion),  how  It  does  this  Is  not  clear  since  the  experts'  opinions  are  shifting 
due  to  the  Delphi  technique.  In  contrast,  seal ! ng/cl uster I ng  gives  a  precise 
measure  of  unaccounted  variance^  Second,  experts  may  have  highly  complex  multi¬ 
dimensional  precept Ions  which  they  cannot  rt  y  dissect  without  aid.  Intro¬ 
spection  may  be  Inadequate  to  abstract  the*-  o.  ceptual  categories  or  dimen¬ 
sions.  Similarly,  an  analyst  could  use  such  techniques  on  htmself  to  learn  what 
categories  or  dimensions  seemed  to  be  Important  In  a  complex,  multidimensional 
problem.  Third,  It  Is  feasible,  at  least  In  theory,  to  obtat n  these  seal t ng/ 
clustering  results  unobtrusively,  at  a  distance,  e.g.,  from  the  writings  of 
Soviet  naval  officers  or  authorities. 

The  latter  application  of  scaling  requires  content  analysis  of  the  co¬ 
occurrence  of  descriptors  with  objects.  For  example,  suppose  Soviet  Admiral  x. 
a  navel  expert,  always  decrlbes  the  Kr.es ti  1  and  Krasta  11  cruisers  with  Identi¬ 
cal  modifiers.  Furthermore,  ass  «e  some  of  these  modifiers  are  used  to  describe 
Karas,  but  none  are  used  to  descr  t  be  Kr  I  yaks.  A  measure  of  s  I  m  1 1  ar  I  ty  can  be 
obtained  by  means  of  the  degree  of  overlap  In  use  of  modifiers  for  these  and 
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other  ships.  These  similarity  measures,  in  turn,  can  be  used  to  determine  the 
degree  to  which  (and  the  categories  or  dimensions  on  which)  Admiral  X  perceives 
these  a.ilp  types  as  similar  or  different  (Rosenberg  and  Jones,  1972;  Rosenberg 
and  Sediak,  1972).  Similarly  an  analyst  might  Investigate  the  perceptual  dimen¬ 
sions  of  Soviet  statements  on  missions  (e.g..  sea  denial,  anti-sea  lines  of 
communication,  etc.),  doctrine  (e.g.,  protection  of  state  interests),  events, 
capabilities,  etc.  That  is,  merely  by  describing  stimuli,  Soviet  spokesmen  are 
revealing  considerable  information  on  the  perceptual  categories  and  dimensions 
they  apply  to  complex  objects  and  events.  This  Information  can  be  abstracted 
from  their  statements  by  analysts  and  aval uated  with  scaling  or  cl  us ter  I ng 
techniques. 


B.  WEIGHTING  DATA 

The  fact  that  people  often  cannot  accurately  report  the  weights  they  attach 
to  data  In  making  estimates  suggests  that  expl icit  "po! Icy  capturing"  asses¬ 
sments  of  analysts  may  assist  them  to  understand  and  improve  their  estimation 
processes.  That  is,  an  explicit  effort  can  be  made  to  model  or  capture  the 
quantitative  elements  of  the  estimation  process  of  the  analyst  Including  data 
weighting. 


Policy  Capturing 

Figure  13  shows  schematical  iy  how  this  can  be  done.  A  control  led  set  of 
data  stimuli  (S)  are  presented  to  the  analyst  and  the  analyst’s  estimative 
response  is  observed  (R).  The  right  side  of  the  diagram  suggests  that  analyst's 
estimation  process.  The  Input  stimuli  and  the  output  response  for  this  process 
must  be  quantified;  t.e.,  they  may  be  qualitative  In  nature  originally  but  they 
must  be  scaled,  coded,  or  rated  by  the  analyst  to  yield  at  least  a  more- 1  ess. 


65 


Figure  13.  General  Estimation  Diagram  Illustrating  Prediction  Analysis 

(on  the  left)  and  Process  Analysis  (on  the  right).  Stimuli  (S^) 
a  common  to  both.* 
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From  Shanteau  and  Phelps,  1977:  258. 


plus-minus  quant  I  tat! ve  est I  mate.  I  f  the  I npu+s  can  be  quantified  di rect I y , 
analyst  coding  of  them  Is  not  needed.  This  process  is  repeated  with  varied 

I 

|  stimul  i  sets,  i.e.,  the  analyst  sees  a  new  set  of  data  stimul  i  and  makes  a  new 

|  estimate. 

\  The  analyst's  weighting  policy  is  capfured  by  using  the  same  coded  data 

\  stimuli  sets  to  mathematically  predict  the  analyst's  estimative  responses,  R. 

Shown  on  the  lett  in  the  diagram.  That  is,  using  the  same  inputs,  S,  we  solve 
for  the  combination  rule  which  makes  the  optimal  responses,  Y,  as  close  as 
possible  to  the  analyst's  responses,  R.  The  combination  rule  found  will  reflect 
a  set  of  objective  weight  values  that  Indicate  the  weights  used  by  the  analyst 
in  his  or  her  estimates.  The  objective  weights  provide  indices  on  the  degree  to 
which  the  analyst  used  each  of  the  input  data  stimuli  dimensions. 

Two  mathematical  policy  capturing  methods  are  commonly  used,  linear 
regression  and  Bayesian  analysis  (Slovic  and  Lichtenstei r  1971).  In  general, 
the  linear  regression  method  has  been  used  more  of"  for  assessing  data 
weights,  but  the  Bayesian  assessment  of  the  "d t agnost i cl ty"  of  data  can  also 
provide  Information  on  the  degree  to  which  analysts  walght  data  (see  Edwards, 
1978). 

If  the  important  dimensions  are  known  on  which  analytic  Judgments  are  made 
(e.g.,  from  a  multt dimensional  scaling  of  analysts'  similarity  Judgments,  see 
above)  analysts  might  simply  be  asked  to  rate  or  rank  the  dimensions  In  impor¬ 
tance.  These  we! ghts  can  then  be  compared  to  the  poll cy  captur 1 ng  object  I ve 
weights  to  determine  the  self-insights  of  the  analysts  Into  their  Judgment  and 
Information  processes. 

Bayesian  Techniques 

Bayesian  estimation  methods  have  been  used  extensively  In  intelligence 
analysts  (see  Slovic  and  Lichtenstein,  1971:  717-721,  and  Slovic,  Fischhoff,  and 
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Lichtenstein,  1977:  25-28,  for  reviews  and  references).  In  general,  these 
methods  heip  analysts  Integrate  probabilistic  data  Into  their  Judgments, 
avoiding  the  tendency  to  underweight  such  data  and  not  adjust  posterior 
probabl  I  (ties  sufficiently.  A  recent  appl  I  cat  I  on  of  Bayesian  methods  to  the 
problem  of  estlamtlng  probabl I ttles  of  a  Middle  East  conflict  was  described  by 
Schweitzer  (In  Heuer,  1978,  ch.  2j.  As  Schweitzer  noted  (p.  19)  these 
techniques  have  been  appl  ted  to  a  variety  of  Intel  I  Igence  estimation  Issues: 
the  likelihood  of  a  North  V I etnamese  offensive  In  1974,  the  probab II  I ty  of  a 
Slno-Sovlet  conf  I  let,  the  chances  of  an  Arab-lsrael  I  war,  and  the  analysis  of 
order  of  battle  data  (p.  13). 

Bayesian  methods  have  also  been  recommended  as  a  means  by  which  the 
estimates  of  different  experts  can  be  effectively  combined.  Hennessey  (1977) 
recommended  a  Bayesian  paradigm  for  the  systematic  cumulation  of  evlaence  from 
related  studies.  He  suggests  this  would  overcome  three  widespread  difficulties 
which  make  research  data  difficult  to  Interpret:  (1)  the  circuitous  and  non  In¬ 
tuitive  logic  of  traditional  statistics,  (2)  lack  of  agreement  (often  latent  and 
Implicit)  among  analysts  on  the  substantive  and  technical  premises  adopted  In 
research  arguments,  and  (3)  the  low  d I agnost  I c I  ty  (weakness)  of  research  data 
for  distinguishing  among  alternative  hypotheses.  Morris  (1974,  1976)  recom¬ 
mended  Bayesian  methods  for  Integrating  the  Judgments  of  experts  Into  a  single 
estimate  and  outlined  a  possible  combination  of  mechanism  for  this  (1976). 

Sample  Size  and  Base-Rates 

Oayeslan  methods  can  also  help  the  analyst  with  a  common  weighting  problem, 
the  tendency  to  overweight  case  data  and  to  underweight  base-rate  data.  By 
successively  updating  prior  probabilities,  the  Bayesian  techniques  "build*’  base- 
rate  data  Into  the  estimation  process.  They  also  tend  to  scale  down 
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overweighted  case  data  by  adjusting  Its  Impact  downward  via  the  prior 
probability  base  rate. 

Simpler  methods  can  help  analysts  give  more  weight  to  groups  of  cases  then 
to  the  most  recent,  vivid,  or  salient  case.  When  case  data  are  coded,  use  of  a 
moving  average  tends  to  "smooth  out"  extreme  data  points  (I.e.,  highly  salient 
or  vivid  cases)  and  adjusts  recent  points  to  reflect  the  recent  base-rates 
(I.e.,  the  span  of  the  moving  average).  Comparing  case  statistics  to  the  cen¬ 
tral  tendency  for  all  cases  (e.g.,  averages)  allows  the  analyst  to  put  the  case 
Into  perspective. 

The  use  of  averaging,  moving  averages,  and  Bayesian  probability  updating 
also  reduces  the  tendency  to  overweight  data  from  extremely  small  samples,  and 
helps  reinforce  the  law  of  large  numbers,  that  estimates  based  on  large  samples 
of  cases  are  more  representative  than  estimates  based  on  small  samples.  The  two 
main  objectives  In  these  methods  are  to  help  the  analyst  to  avoid  under-  or 
over-react  I ng  to  a  single  piece  of  Information,  and  to  use  the  cumulative 
Information  contained  In  base-rate  data. 

Redundancy  and  Variance 

Analysts  can  control  the  tendency  to  overweight  redundant  cues  by  computing 
the  correlations  between  them  and  reducing  the  weight  attached  to  a  highly 
redundant  (correlated)  variable.  Similarly,  the  analyst  should  note  the  vari¬ 
ability  I  n  h  I  s  data  and  reduce  his  weighting  of  a  cue  which  stops  vary  I  ng  and 
becomes  static. 

C.  CHARACTER  I Z I NG  OATA 

A  variety  of  memory  biases  ware  discussed  which  affect  the  character Izatlon 
of  data  and  Its  organization  Into  categories,  factors,  classes,  and 
general Izatlons.  Problems  were  noted  that  result  from  attempts  to  Impose 
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"either/or"  categories  on  events  or  objects  which  share  similarities  as  well  as 
differences.  The  tendencies  to  place  events  and  objects  Into  taxonomies  was 
discussed  and  the  possible  biases  that  may  result  were  described.  A  variety  of 
anaiytlc  methods  and  paradigms  can  help  offset  these  difficulties. 

MAlflgJtoOKO 

The  remarkable  ability  of  naval  analysts  to  recall  information  is  due 
largely  to  their  complex  conceptual  schemas  for  perceiving  relationships  between 
aspects  of  the  Soviet  Navy.  These  schemas  allow  analysts  to  efficiently  code, 
store,  and  retrieve  Information.  In  general,  however,  the  schematic  rules  which 
govern  these  processes  are  tacit  and  Implicit  In  the  analysts*  narrative 
estimates.  One  Implication  of  this  Is  that  analysts  may  disagree  becasue  their 
different  schemas  lead  to  different  perceptions  and  memories  of  the  sare  or 
similar  data  sets.  Since  the  schematic  rules  for  processing  these  data  sets  are 
not  explicit,  such  sources  of  differences  cannot  be  explicitly  determined. 

This  suggests  that  If  analysts  make  their  schematic  coding  and  storage  of 
I  n  for  mat  1  on  more  exp  1  I  cl  t,  the  job  of  detorm  I  n  I  ng  the  sources  of  d  1  f  f  erences 
among  analysts  In  categorizing  data  would  become  easier.  Analysts  would  be  able 
to  compare  categorization  systems  and  contents  explicitly,  as  well  as  comparing 
their  conclusions. 

The  tremendous  growth  of  data  base  management  systems  and  management  Infor¬ 
mation  systems  In  business,  admlnstratlon,  and  government  reflects  the  apprecia¬ 
tion  of  the  need  for  extensive  and  flexible  means  for  accurately  coding,  stor¬ 
ing,  retrieving,  and  organizing  Information.  Kfhl  le  these  systems  tend  to  fcs 
most  often  applied  to  quantitative  data,  they  can  also  be  applied  to  coded  event 
data  or  content  analysis  data.  Kirk's  (1980)  content  analytic  data  on  Soviet 
political  elite  speeches,  for  example.  Is  stored  In  a  data  base  management 
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system,  greatly  facilitating  extensive  data  retrieval  or  manipulation  and  empl- 
rtcal  Investigations. 

I  n  ear  I  I  er  work  by  Stech  (1981,  ch.  4)  an  examp  le  was  used  to  demonstrate 
that  a  naval  analyst's  conclusion  was  based  on  biased  sampling;  Stech  used  that 
analyst's  explicit  coding  of  events  (diplomatic  port  visits).  Had  these 
explicitly  coded  data  not  been  available,  we  could  not  have  determined  that  the 
analyst's  conclusion  was  based  on  a  biased  sample. 

Naval  analysts  themselves  complain  of  being  unable  to  deduce  the  Implicit 
coding  schemes  of  other  analysts.  For  example,  McConnel  (SN1  612)  wrote: 

McGwire  and  Er!ckson...count  on  their  subjective  impressions 
of  "tone"  and  "thrust"  to  tell  them  that  Groshkov  Is  an 
advocate.  I  respect  the  conviction  behind  this  approach, 
and  perhpas  I  avoid  tt  only  because  I'm  not  good  at  It... It 
has  been  my  experience  that  others  have  a  so-so  record  In 
this,  too,  as  often  as  not. 

Mak I ng  subjective  impressions  I nto  exp  I  left  cod  I ng  rules  Is  often  a  fairly 
stra  i  ghtforward  (If  not  always  simple)  process  that  Is  In  keeping  with  the 
scientific  requirement  that  subjective  Impressions  be  replaced  by  quantitative 
measurements.  Such  coding  greatly  helps  analysTS  determine  the  validity  of 
their  own  and  others  Impressions,  as  wsl I  as  facilitating  accurate  recall. 

Event  coding  and  content  analysis  also  aid  the  analyst-  tn  efforts  to  avoid 
"sel ect I ve  retr I eva I ",  by  f ac 1 1  t  tat  I ng  the  reca II  of  1  he  actua I  original  data 
rather  than  a  retrospect  I ve  reconstruct  I  on  of  It.  The  tendency  to  "construc¬ 
tively  remember"  events,  a  process  affected  by  hindsight  and  a  variety  of  memory 
b I ases.  Is  minimi zed  If  the  analyst  can  quickly  ascerta (nail  the  other  cases 
that  fall  Into  a  given  category,  or  that  compare  favorably  on  sal  lent  dimensions 
to  a  case  In  point.  Mechanical  coding  systems  provide  the  analyst  this  ability 
to  organize  and  manage  stored  Information  and  help  minimize  the  analyst's  need 
to  rely  on  I imlted  and  fal 1 Ible  memory. 


Another  reason  analysts  should  make  greater  efforts  to  code  their  Impres¬ 
sions  Is  the  tendency  for  memory  to  distort  psychophysical  tmpresslons  of  ex¬ 
treme  cases.  Extremes  (biggest,  worst,  smallest,  etc.)  are  well  remembered  as 
being  extreme,  but  their  recalled  dimensions  tend  to  be  less  extreme  than  the 
original  dimensions.  Psychol og I  sts  have  made  considerable  progress  In  devel¬ 
oping  methods  for  scaling,  scoring,  rating,  or  otherwise  measuring  psychological 
sensations  (e.g..  see  Anderson,  1979).  Analysts  might  employ  such  functional 
measurement  techniques  to  record  the  magnitudes  of  their  original  Impressions  so 
that  current  Impressions  could  be  maintained  for  later,  accurate  comparisons 
with  future  Impressions.  Anderson  (1979)  reported  results  from  a  study  by  Leon 
that  are  relevant  to  the  present  study.  Adults*  and  childrens'  Impressions  of 
"naughtiness"  were  recorded  for  various  Incidents  that  varied  In  terms  of  the 
degree  of  severity  of  damage  done,  and  In  the  Intent  of  the  person  doing  the 
damage.  Naughtiness  was  found  for  both  adults  and  children  to  be  a  linear 
function  of  both  severity  and  of  Intent,  but  children  weighted  Intent  less  and 
damage  more  In  determining  naughtiness  than  did  the  adults.  It  Is  highly  likely 
that  many  naval  analysts  conceive  of  threats  as  being  a  function  of  capabilities 
and  Intentions.  It  would  be  very  useful  to  try  these  psychological  techniques 
to  determine  how  analysts  emphasize  theso  two  components  of  threat  for  various 
specific  Issues  and  questions,  and  to  be  able  to  record  their  Impressions  over 
time  In  some  form  of  comparable  metric. 

As&gaalna  PratatYags.-Cataggriaa,,  and,  £JJngnalaoa 

Stech  (1981)  argued  that  people  conceptualize  complex  events  and  objects  In 
"fuzzy"  categories  with  loose,  overlapping  boundaries,  rather  than  In 
■elther/or*  pigeonholes.  This  Implies  that  category  membership  Is  a  matter  of 
"f  am  I  ly  resemb  I  ance"  and  that  stimuli  are  coded  In  terms  of  many  dimensions 
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relative  to  a  central  prototype.  Analysts  wouia  oe  oes?  »i  . 

Impressions  could  be  coded  on  the  natural  dimensions  they  themselves  use  to 
perceive  events  or  objects,  and  if  their  subjectively  meaningful  concepts  of 
family  resemblance  and  prototypical  I ty  were  the  bases  for  the  coding  and  content 
analyzing  of  data.  Rather  than  Imposing  arbitrary  coding  dimensions,  or  content 
analysis  categories  on  the  analyst,  recent  psychological  research  suggests  that 
It  Is  possible  to  determine  explicitly  the  natural  categories  or  dimensions  the 
analyst  uses.  These  natural  categories,  or  dimensions  once  made  explicit,  could 
then  provide  the  analyst  the  ability  to  systematically  characterize  his  or  her 
Impressions  without  grossly  distorting  the  analyst's  cognitive  process.  This 
psychological  research  also  suggests  that  the  natural  categories  or  dimensions 
are  rarely  comp  I etel y  explicit  In  the  analyst's  mind  before  such  assessment. 
These  techniques  may  thus  help  make  the  analyst's  methods  and  assumptions  more 
accessible.  In  keeping  with  Heuer's  (1978)  recommendations. 

Objects  or  other  stlmul  I  can  be  considered  to  have  a  set  of  features  or 
attributes.  A  person's  total  data  base  concerning  any  given  object  Is  rich  In 
content  and  complex  In  organization  and  form.  It  Includes  features  of  appear¬ 
ance,  meanings,  functions,  relationships,  history,  and  all  other  properties  that 
are  known  or  can  be  deduced.  When  faced  with  a  particular  analytic  task  (e.g.. 
to  Identify  the  object,  or  determine  Its  similarity  or  dissimilarity  from  other 
objects)  people  extract  and  compile  from  their  data  base  a  limited  I  I  st  of 
relevant  features  to  perform  the  task  (Vversky,  1977). 

There  are  two  approaches  that  can  be  taken  to  relate  objects  to  one  an¬ 
other.  One  measures  the  distances  between  the  features  of  objects  In  a  geome¬ 
tric  sense.  The  other  considers  the  overlap  of  common  features  relative  to 
uncommon  features  In  a  set-theoretic  sense.  Which  of  these  two  approaches  are 
used  to  Identify,  quantify,  and  organize  prototypes,  categories,  or  dimensions 
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*111  depend  on  the  objects,  the  task,  or  both  (Stech  and  TversKy,  1977;  Shepard, 
1980).  Some  problems  facing  naval  analysts  have  a  natural  dimensional  struc¬ 
ture,  e.g.,  estimating  the  severity  and  probability  of  threats.  Others  have 
hierarch  leal  structure  that  may  ref  I  ect  an  evol  utlonary  process  In  which  the 
objects  all  have  an  Initial  common  structure  and  later  develop  additional  dis¬ 
tinctive  features.  An  example  of  the  latter  might  be  the  classification  of 
Soviet  Navy  platforms. 

Soviet  naval  analysts  have  no  widely  accepted  method  for  classifying  the 
Soviet  general  purpose  navy  Into  Its  component  missions.  Thorpe  (NPSP.  ch.  8) 
attempted  to  develop  such  a  method  using  Delphi  techniques.  A  variety  of 
problems  occur  when  OeiphI  techniques  are  used  (see,  e.g.,  Morgenstern,  Knorr, 
and  Helss,  1973:  23-26).  For  example,  the  range  of  expert  opinion  converges 
sharply,  although  there  Is  no  normative  reason  why  such  convergence  should  lead 
to  greater  estimative  accuracy.  The  central  tendency  of  expert  judgments  often 
shifts,  but  In  a  manner  that  has  no  discernible  relation  to  the  new  Information 
available  to  experts.  That  Is,  It  Is  unclear  whether  Delphi  Is  an  appropriate 
mechanism  for  Information  I ntegratlon.  OeiphI  Is  also  costly  and  time- 
consuming. 

Clustering  and  scaling  techniques  offer  a  far  more  promising  method  of 
categorizing  Judgments  by  experts  and  analysts  than  the  questionable  and  costly 
OeiphI  method.  For  example,  to  determine  how  analysts  classify  Soviet  general 
purpose  platforms  In  terms  of  wartime  missions,  Thorpe's  objective,  one  could 
follow  the  procedure  used  by  Rosch  and  Mervls  (1975)  to  classify  vehicles  (see 
also  Tversky,  1977:  338).  Analysts  would  be  asked  to  list  all  the  possible 
wartime  missions,  or  ai ternatl vel y.  one  could  give  all  analysts  a  list  of 
missions,  as  Thorpe  did.  For  each  wartime  mission  and  each  plarform,  analysts 
would  be  asked  to  list  those  features  of  the  platform  that  were  relevant 


74 


(posi+tvely  or  negatively)  for  that  mission.  These  Itsts  provide  the  wartime 
mission-relevant  features  of  eacn  Soviet  platform  ror  each  analyst  or  expert.  A 
master  catalog  (sometnlng  like  Jane's  Ships)  of  features  and  attributes  for  each 
platform  could  be  made  available  to  all  analysts,  listing  weapons,  electronics, 
beam,  draft,  propulsion,  etc.,  to  refresh  the  analyst’s  Impressions  and  to  serve 
as  a  codebook  for  the  listings. 

It  Is  then  possible,  using  the  I  Ists  of  features,  to  determine  for  each 
pair  of  platforms  the  number  of  common  and  distinctive  features.  From  these 
data  It  Is  possible  to  predict  with  high  accuracy  the  analysts'  ratings  of 
similarity  between  platforms  given  any  wartime  mission,  using  the  data  on 
shared  and  nonshared  features  and  (derived  or  obtained)  ratings  of  similarity, 
clustering  programs  can  be  used  to  determine  a  hierarchical  clustering  diagram. 
Th I s  diagram  prov I  des  a  deta Med  classification  of  each  p I  atf orm's  perce I  ved 
capabilities,  relative  other  platforms,  to  perform  each  wartime  mission.  The 
diagrams  created  by  these  programs  reveal  the  main  cognitive  categories  used  by 
the  analysts  to  make  these  Judgments.  That  Is,  not  only  are  Soviet  Navy  plat¬ 
forms  categorized  by  wartime  mission,  as  Thorpe  attempted,  but  the  clustering 
algorithms  allow  us  to  determine  the  main  dimensions  the  analysts  used  to  make 
these  Judgments.  The  latter  Information  cannot  be  derived  from  Thorpe's  Delphi 
method.  The  net  result  from  the  clustering  approach  would  be  a  classification 
diagram  for  each  wartime  mission  showing  how  each  Soviet  platform  compares  with 
a  I  I  other  p l atf or ms  t  n  accompl I sh I ng  that  mission,  how  the  analysts  grouped 
platform*  of  similar  capabilities  ,  and  how  (and  why)  Individual  platforms  and 
groups  of  platforms  differ  from  each  other  In  performing  that  mission. 

These  feature  analysts  techniques  also  allow  us  to  determine  for  each 
wartime  mission  the  prototyplc  features  for  that  mission.  A  measure  of  family 
resemblance  (distance  from  the  prototype)  for  each  platform  can  be  derived  from 
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these  data  which  allows  direct  scaling  of  the  rankings  of  Individual, 
multipurpose  naval  ships  and  aircraft  among  a  variety  of  missions.  That  is,  a 
ship  with  an  extremely  close  resemblance  to  the  prototype  for  a  particular 
mission  would  be  scaled  to  have  a  high  ranking  for  that  mission. 

Thorpe's  analysis  requires  that  if  a  platform  has  a  high  weight  (percent) 
for  one  mission,  it  must  have  a  low  weight  for  other  missions  (I.e.,  percents 
can  only  add  up  to  100).  This  forces  "ei ther/or"  distinctions  Into  what  are 
actua I  I y  "and/both"  Judgments.  For  example,  a  modern  Soviet  ship  such  as  the 
Kresta  1 1  may  be  a  far  better  anti  ship  platform  that  the  obsolete  Krunnv.  but  In 
Thorpe's  method,  the  former  gets  20  percent  for  ant  I  ship  (because  of  Its  heavy 
weight  for  A5W)  while  the  latter  gets  a  70  percent.  While  Thorpe's  method  may 
be  necessary  for  the  economic  analyses  he  performs,  it  Is  probably  highly 
misleading  as  a  reflection  of  analysts'  categor t zat I  on  of  platforms  and 
missions.  In  Thorpe's  method  a  Soviet  choice  of  an  obsolete  Kruonv  over  a 
Kresta  1 1  would  be  scored  as  an  increased  Soviet  Navy  emphasis  on  the  ant  I  ship 
mission  simply  because  the  Kruony .  ineffective  as  it  may  be  at  ant  I  ship  warfare, 
is  even  less  capable  of  ASW.  This  seems  to  be  an  absurd  conclusion  no  naval 
analyst  would  make.  A  more  meaningful  measure  of  mission  priortty  would  be  to 
measure  the  fatally  resemblances  of  each  year's  new  Soviet  platforms  to  the 
mission  prototypes.  This  would  allow  mission  comparisons  without  imposing  the 
unreasonable  trade-off  logic  of  percentage  estimates. 

Intention  Categories 

The  classification  of  Soviet  naval  platforms  provides  a  ready  application 
of  modern  categor icatlon  methods  to  Soviet  capab l I i t I es.  The  many  overlapping 
and  nonover I app i ng  features  of  ships  provide  an  easily  quantified  basis  for  sucn 
analysis. 
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With  appropriate  adaptations  the  same  categorization  methods  might  be 
applied  to  I ssues  of  I ntent Ions  as  we  1 1 .  For  examp  I  e,  naval  analysts  develop 
models  of  Soviet  naval  behavior  In  various  situations  and  cases  (e.g.. 
McConnell,  In  Qismukes  and  McConnell,  ch.  7).  It  should  be  possible  to  assemble 
lists  of  the  features  of  these  situations  and  cases.  Analysis  of  these  features 
could  then  determine  the  dimensions  on  which  analysts  perceive  Soviet  Intentions 
as  varying.  The  dimensional  structures  obtained  by  analysis  of  overlapping  and 
nonoverlapping  features  could  be  evaluated  by  assessing  the  analyst's  percep¬ 
tions  of  s I m 1 1 arl ty  and  d I f f erences  between  cases.  These  I atter  data  can  be 
used  to  create  a  dimensional  space  that  should  correspond  to  tne  dimensions 
obtained  from  features  analysis. 

In  short,  modern  psychological  techniques  allow  a  quantitative  assessment 
of  semantic  and  perceptual  dimensions  or  categories.  The  natural  categories  can 
be  determined  and  then  used  as  the  basis  for  expl  left  coding  and  quantitative 
analyses.  Because  the  dimensions  and  categories  are  obtained  from  the  analyst's 
own  cognitive  relations,  they  tend  to  yield  Intuitively  useful  classifications, 
which,  however,  are  not  Intuitively  obvious  and  often  cannot  be  obtained  by 
other  methods.  An  Important  consequence  Is  that  the  quantitative  coding  based 
on  these  techniques  Is  likely  to  be  high  In  qualitative  meaning  to  the  analyst. 

Easjac.Aaabala.  IxiUjaifllLLganca 

Several  recent  studies  have  attempted  to  quantify  the  factors  analysts  use 
In  analysis.  Oahlgren  (In  Heuer,  1978.  ch.  5)  translated  a  complex  theory  of 
International  political  violence  i-'to  about  fifteen  separate  social  and  societal 
factors.,  or  variables,  and  various  relationships  among  them.  A  panel  of 
Intelligence  analysts  assigned  numerical  values  to  each  of  the  theory's 
variables  and  the  median  scorings  across  analysts  for  each  variable  were  used  to 
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evaluate  the  theory.  Note  that  Oahlgren  did  not  derive  the  variables  from  the 
analysts'  cognitive  relations,  but  frcm  a  theory  of  political  violence. 

One  Intelligence  application  of  the  features  similarity  techniques  de- 
scr I  bed  above  was  Kent  and  W II ey's  (in  Heuer,  1978,  ch.  8)  use  of  mu  1 1!  d I men- 
slonal  scaltng  to  determine  voting  blocs  In  the  United  Nations.  While  useful 
and  suggestive,  and  we  1 1 -received  by  the  analysts,  the  results  of  this  Investi¬ 
gation  are  ambiguous  largely  because  the  wrong  methodology  was  applied.  Voting 
bloc  analysis  Is  Inherently  a  clustering  or  grouping  problem,  rather  than  a 
dimensional  problem.  Rather  than  multidimensional  scaling,  Kent  and  Wiley 
should  have  applied  a  clustering  or  tree-bul I  ding  technique,  and  attempted  to 
develop  a  voting  bloc  taxonomy.  A  cluster  or  tree  diagram  would  be  much  more 
representative  of  the  voting  subgroupings,  which  Kent  and  Wiley  set  out  to  find, 
than  are  the  seal  I ng  d I agrams  they  derived  from  voting  s I m 1 1 ar I ty  data.  The 
main  point,  however.  Is  that  Intelligence  analysts  have  used  these  psychological 
methods  on  features  similarity  data  with  some  success. 

Frledhela  and  Hehn  ( SNP.  ch.  185  used  content  analysis  of  United  Nations 
documents  on  the  Law  of  the  Sea  to  determine  Soviet  positions  on  various  Issues 
under  negotiation  In  the  Law  of  the  Sea  talks.  For  each  Issue  (e.g.,  rights  of 
transits  through  straits)  the  frequency  of  various  themes  mentioned  (e.g..  free 
transits  with  exceptions  and  limits)  provided  a  score  for  that  theme.  By  scor¬ 
ing  themes,  Frledhetm  and  Hehn  were  able  to  compare  the  view  of  the  United 
States,  the  USSR,  and  Admiral  Gorshkov  on  several  Law  of  the  Sea  Issues.  Fried- 
helm  and  Hehn  were  also  able  to  predict  how  often  nations  would  favorably 
mention  various  .‘hemes  by  regressing  character  I  st  l  cs  of  the  countries  (e.g., 
geographical  characteristics,  economic  Interest,  etc.)  against  the  thematic 
scores.  This  technique  allows  the  analysts  to  determine  factors  which  seem  to 
aeke  a  difference  In  bargaining  positions.  For  example,  the  Important  factors 
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for  the  USSR  position  on  various  issues  were  (p.  354):  membership  in  the  East 
European  caucusing  group,  major  merchant  fleet,  Blue  Water  Navy,  major  fishing 
state,  distant  water  fishing,  straits  state,  broad  shelf,  major  mineral  pro¬ 
ducer,  major  oil  producer,  and  offshore  oil  producer. 

Several  :iaval  analysts  have  developed  methods  aimed  at  assessing  the 
Intentions  of  Soviet  ship  designers.  These  methods  take  an  approach  that 
Includes  explicit  decomposition  of  destgn  requirements  and  ship  features  and  an 
att«mot  to  logical ly  re! ate  one  set  to  rhe  other.  While  neither  of  the  two 
methods  described  beiow  used  the  psychometric  features  analysis  methods 
described  above,  both  suggest  that  systematic  use  of  features  data  is  not 
foreign  to  naval  analysts. 

Kehoe  ( SN 1 .  ch.  19)  presented  a  methodology  for  assessing  the  factors 
underlying  warsnips  design  which  is  quite  compatible  with  the  psychometric 
approach  outlined  above.  He  determines  the  s ■  ip  mission  requirements  for 
payload  and  performance  in  terms  of  various  factors:  hui  !  size,  seakeeping, 
propulsion  speeds,  cruising  endurance  range,  habitability,  payload,  electronics, 
and  weapons.  These  are  thv  ^ges  of  features  we  suggested  above  that  analysts 
could  assess  and  list  for  Mous  naval  platforms.  In  his  chapter,  Kehoe 
compares  trends  in  these  features  for  Soviet  and  U.S.  ship  types  over  time. 
Kehoe  had  experts  evaluate  the  trend  data  and  rank  the  major  design 
characteristics  Into  prior  tty  order,  or  as  Kehoe  put  it  (p.  380): 

...wnich  of  these  design  characteristics  appeared  to  "get 
the  biggest  piece  of  the  cake"  in  the  ship  design  process? 

The  experts  determined  that  Soviet  ship  designers  emphasize  factors  in  the  same 
priority  as  did  the  United  States  In  building  World  War  II  vintage  ships. 

Another  means  of  determining  priority  was  not  used  by  Kehoe,  but  has  been 
used  In  other  fields  to  determine  critical  features  of  technological  change. 
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This  is  to  regress  various  design  features  to  predict  some  feature  of  technical 
merit.  One  such  feature  is  Initial  operating  capability  date,  which  is  a 
surrogate  for  modernity.  The  question  then  becomes  which  design  factors  seem  to 
be  driving  Soviet  ship  developments?  A  similar  technique  was  used  by  Alexander 
and  Nelson  (1972)  to  assess  the  factors  influencing  Soviet  and  U.S.  aircraft 
turbine  engine  designs. 

Kehoe  evaluates  his  data  on  design  factor  trends  in  terms  of  various  Soviet 
missions  (e.g.,  sea  denial).  it  wou!d  be  interesting  to  use  the  features 
analysis  methods  outlined  above  to  determine  natural  taxonomies  or  dimensions  of 
Soviet  ships,  and  to  see  If  similar  mission  categories  or  dimensions  emerged. 

Meier  SNI.  cn.  20)  describes  a  different  method  aimed  at  the  same  objec¬ 
tive  as  Kehoe's,  determining  the  relative  priorities  given  by  the  Soviets  to 
firepower,  sustained  combat,  command  and  control,  speed,  endurance,  seakeeping, 
protection,  and  personnel  support.  Meier's  method  Is  a  reverse  engineering 
process  which  uses  an  iterative  computer  program  that  attempts  to  determine 
feasible  design  requirements,  standards,  and  practices  from  the  features  of  the 
finished  ships. 

0.  ASSESSING  COVARIATIONS 

Stech  (1981)  noted  the  tendency  of  people  to  overestimate  the  strength  of 
theory-driven  covariations  and  to  neglect  or  underestimate  the  strength  of  data- 
driven  covariations.  We  also  found  this  tendency  seemed  to  extend  to  naval 
analysts.  Obviously,  the  most  direct  remedy  for  these  problems  Is  to  Increase 
the  use  of  quantitative  measures  of  co-occurrences.  Such  measures  should  be 
habitually  applied  to  any  data  that  coul  d  conceivably  be  re  I  a  ted.  This  will 
tend  to  reduce  beliefs  In  spurious  theory-driven  correl atl ons,  because  accumu¬ 
lating  evidence  will  fail  to  support  the  theory,  and  to  drew  attention  to 
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unsuspected  relationships.  Only  some  of  the  observed  correlations  In  data  sets 
will  be  meaningful:  some  will  be  spurious  statistical  noise,  and  some  will  be 
statistically  reliable  but  un Interpretab  I e.  None  of  these  are  major  problems. 
Perhaps  the  main  problems  are  to  encourage  analysts  to  give  data-driven  pat¬ 
terns  of  co-occurrence  serious  consideration,  and  to  reconsider  theories  which 
are  unsupported  by  data  patterns. 

Actuarial  Models  and . Backcastt ng 

Referring  to  Figure  13  we  can  note  the  optimal  response,  Y,  given  on  the 
left  side  of  the  diagram.  This  is  the  best  prediction  possible  given  the 
objective  values,  X,  of  the  stimulus  Inputs,  S.  For  example,  a  naval  analyst 
may  be  abie  to  measure  (X)  severai  features  (S)  of  the  Soviets’  building  program 
for  a  particular  ship  class.  The  analyst  may  regress  these  against  actual 
production  figures  In  the  past  (criterion  values)  to  determine  the  optimal 
combination  rule.  This  rule  can  then  be  used  to  predict  future  output,  Y.  This 
process  I s  I abel ed  actuar I al  and  as  was  outl I ned  I n  Chapter  4,  the  actuar I al 
prediction  process  Is  consistently  superior  to  the  process  shown  on  the  right  In 
Figure  13,  the  cl  Ini  cal  process.  That  Is,  when  estimators  attempt  to  Intui¬ 
tively  integrate  Information  on  the  Inputs,  S,  using  a  psychological  combination 
rule,  they  are  less  accurate  predictors  than  an  actuarial  model  of  the  same 
Judgments. 

This  suggests  that  analysts  make  greater  use  of  actuarial  models  In 
formulating  and  revising  their  theories  about  covariations.  We  will  have  more 
to  say  about  these  techniques  below  under  ’’Improving  Predictions”.  One  of  the 
advantages  of  actuarial  methods  Is  that  they  force  the  analyst  to  assess  the 
relationships  between  criterion  value?  a r'*  Input  data.  Actuarial  methods  force 
the  analyst  to  consider  the  strength  of  theory-dr iven  covariatton  hypotheses  In 
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I Ight  of  past  data.  If  the  theory-driven  relationship  Is  absent  In  the  past, 
the  analyst  must  reconsider  using  the  theory  as  a  basis  for  future  predlcltons. 
This  use  of  actuarial  techniques  and  past  data  to  check  suspected  theoretical 
relationships  is  a  form  of  "backcast! ng",  I.e.,  an  attempt  to  correctly  estimate 
when  true  va  I  ues  areal  ready  know  n.  I  f  a  theory  cannot  successful  I  y  fit  past 
events.  Its  validity  for  the  future  is  questionable.  If  the  theory  consistently 
over-  or  under-predicts  past  criterion  values,  the  analyst  can  made  corrections 
to  "debias”  the  error,  and  thus  improve  the  theory's  accuracy. 

The  actuarial  formula  provides  the  optimal  predictions  given  the  Input 
data,  but  It  may  not  be  as  useful  or  interpretable  as  the  analyst's  theory.  In 
other  words,  the  analyst  may  wish  to  retain  and  improve  his  or  her  theory  (which 
is  heuristic  and  interpreted  I e),  while  using  the  actuarial  formula  for  making 
predictions.  As  the  theory  comes  to  resemble  the  actuarial  formula,  the  former 
becomes  more  accurate  as  a  predictor,  and  the  latter  becomes  more  Interpretable. 

In  effect,  research  on  i.'tnlcal  and  actuarial  Judgment  suggests  that 
estimators  should  attempt  to  assess  covariations  between  Input  data  and 
criterion  values  If  an  actuarial  method  is  possible.  When  actuarial  methods  are 
possible  they  provide  the  optimum  description  of  the  covariations  between  Input 
data  and  criteria,  and  greatly  simplify  assessment  of  covariation. 

It  Is  Important  to  note  that  the  expert  Judge  must  specify  what  the  Input 
variables  are  to  be.  Actuarial  model ing  itself  cannot  determine  what  Input 
st  I  mu  I  I  shou  I  d  be  const  dered  as  predictive  of  the  cr  I  ter  Ion  values.  Both  the 
analyst's  theory-driven  hunches  and  empirical  data-driven  search  for  possible 
correlations  can  provide  clues  for  the  inputs  to  the  actuarial  methods.  Only 
the  naval  ana  I ysts  can  know  what  variables  are  I Ikely  to  be  worth  check  I ng  as 
possible  predictors  of  future  Soviet  behavior.  The  actuarial  method  Is  the 
optimal  means  of  asses Ing  these  hunches. 
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When  criterion  values  are  known  or  knowable,  as  In  Soviet  ship  Inventories, 
It  Is  possible  to  fit  regressions  of  Input  data  on  the  criterion  values  to  but  Id 
actuarial  prediction  models.  Analysts,  however,  are  often  called  upon  to 
estimate  values  for  which  there  are  no  clear  criterion  values,  e,g.,  the  level 
of  Soviet  threat.  In  these  cases  analysts  judgmental  I y  provide  the  measure  of 
the  criterion  values,  deciding  ,  e.g.,  whether  the  Soviet  threat  Is  high  or  low. 
No  objective  criterion  values  exist  for  such  cases  against  which  to  compare 
predictions,  I. a.,  the  Y  or  R  In  Figure  13  do  net  correspond  to  any  unambiguous 
measures. 

Th  t  s  I ack  of  cr I  ter  Ion  values  does  not  el  I  m I nate  the  poss I b 1 1  I  ty .  or  the 
need,  for  systematic  methods  of  assessing  covariations.  Psychologists  have 
determined  that  linear  models  which  fit  regression  equations  to  past  clinical 
Judgments  (t.e.,  the  R's  In  Figure  5.1),  can  be  used  to  replace  the  clinical 
Judgment  process.  Such  models  outperform  clinical  Judges  because  they  eliminate 
variations  In  human  reliability.  Reliability  places  an  upper  limit  on  validity. 
If  reliability  Increases,  greater  accuracy  Is  possible. 

This  technique  of  modeling  the  Judge's  Judgments  and  then  using  the 
Judgments  of  the  model  has  been  labeled  "bootstrapping",  t.e.,  pulling  the  Judge 
up  by  his  bootstraps  (Dawes,  1971;  Goldberg,  1970).  Bootstrapping  will  Improve 
Judgments  si Ightiy  under  almost  any  real Istlc  task  conditions  and  It  can  be 
applied  blindly,  In  cases  where  criterion  Information  Is  absent  or  vague,  with 
the  expectation  that  the  predlcitlons  made  will  be  Improved  (Camerer,  1  980). 
Futhermore,  as  long  as  the  regression  model  of  the  judge  determines  the  Input 
variables  for  the  prediction,  determining  the  exact  weights  used  by  the  Judge  Is 
not  necessary;  equal  weights  are  about  as  good  (Dawes  and  Corrigan,  1974).  in 
other  words,  the  weighting  parameters  of  the  bootstr  app  I  ng  model  need  not  be 
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very  specific  once  the  right  variables  are  identified.  The  key  again  is  knowing 
which  variables  to  try  and  the  only  real  istic  source  for  these  is  the  expert. 
Once  the  naval  analyst  identifies  the  variables  that  seem  important  for  making 
an  estimative  Judgment,  the  bootstrapping  method  can  best  determine  the  actual 
covariation  between  those  variables  and  the  expert’s  Judgments.  These  variables 
can  then  be  combined  linearly  with  equal  weights  to  estimate  the  expert’s  future 
Judgments  more  reliably  (and  thus  more  accurately)  than  could  the  expert. 

There  are  obviously  impl Icattons  for  prediction  in  these  actuarial  and 
bootstrapping  models,  but  the  point  for  this  section  Is  that  they  provide 
systematic,  data-driven  means  to  appraise  suspected  covariations  that  eliminate 
the  problems  of  theory-dr i ven  covariation  appraisal. 

Environmental  Effects 

Since  expert  analysts  must  play  a  central  role  in  selecting  the  variables 
for  use  In  actuarial  or  bootstrapp i ong  models,  It  Is  Important  to  reiterate  the 
point  that  people  are  often  insensitive  to  environmental  effects  such  as  regres¬ 
sion  or  base  rates.  That  Is,  analysts  are  uni  ikely  to  adequately  attend  base 
rate  variables,  and  are  likely  to  overattend  case-specific  variables  (Einhorn 
and  Hogarth,  1978).  In  particular,  to  Improve  covariation  assessments.  It  Is 
necessary  to  Include  data  on  disconf irming  events  as  well  as  on  positive  hits. 
That  is,  analysts  should  be  especially  careful  to  collect  and  record  data  on  ‘at 
least)  all  four  cells  of  the  fo*.  fold  present-absent  cross-tab  I u I  at  Ions  that 
determine  the  relation  between  an  Indicator  variable  and  a  predicted  variables. 

One  means  used  frequently  by  naval  analysts  to  display  base  rate  data  Is 
the  time  trend  I  Ine,  l.e.,  a  graph  of  data  over  time.  For  example,  Kehoe  ( SN1 , 
ch.  18)  displayed  data  on  various  capab  1 1  I  ti  es  of  Sov  let  and  U.S.  sh !  p  type?* 
over  time  to  demonstrate  changing  trends  In  various  features.  On  The  other 
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hand,  merely  displaying  a  base  rate  does  not  mean  the  analyst  will  use  the  base 
rate  In  estimates  (e.g.,  see  Ivanoff  and  Murphy,  NPSP:  149,  Figure  32). 

O’Leary  and  Cop  I  I n  (1575,  ch.  4)  presented  a  detailed  case  study  of  how 
careful  cross-tabul ations  of  data  assisted  State  Department  analysts  to  make 
Judgements  about  covariation  between  military  expenditures  and  other  variables 
In  Latin  America.  Although  the  State  Department  analysts  made  extensive  use  of 
quantitative  data,  they  did  not  exp  I IcI t| y  assess  b I var I  ate  cor re  I  at  I  on  s  to 
aval uate  thel r  theories  or  forecasts.  O’Leary  and  Copl I n  showed  that  several 
postu I ated  relationships  between  mill tary  arms  acq uisttlons  and  other  l  ess 
quantitative  factors  (e.g.,  economic  conditions,  role  of  the  military  In  govern¬ 
ment,  need  for  Internal  defense)  could  be  directly  assessed  with  cross-tabu  I  a- 
tions  or  correlation  analysis.  These  techniques  were  applied  to  cross-sectional 
data  (I. a.,  a  group  of  Lattn  American  countries)  as  wel  I  as  to  data  on  Indivi¬ 
dual  nations.  The  covariation  assessments  were  able  to  confirm  several  of  the 
State  Department  analysts'  theories,  as  well  as  showing  that  some  theoretical 
relations  d t  d  not  exist  In  hi stor I  cal  data.  O'Leary  and  Cop  I ! n  descr I  bed  (p. 
1*3)  this  covariation  "backcast I ng"  exercise  as: 

...one  viable  way  of  Incorporating  clearly  defined 
variables,  quantitative  techniques  of  analysis,  and  the 
Important  discriminating  character  of  the  Latin  American 
nation,  all  of  which  appear  to  be  essential  to  making 
accurate  explanations  and  predictions  about  changes  In  Latin 
American  military  spending. 

The  main  Implication  of  the  O’Leary  and  Copl In  study  Is  that  quantitative 
covariation  assessment  methods  can  be  applied  even  when  the  factors  Involved  are 
largely  qual Itattve. 
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£.  CAUSE  AND  EFFECT  ASSESSMENT 


The  problems  analysts  may  encounter  In  generating  cause  and  effect 
explanations  of  Intentions  were  revleweo  In  Stech  1981.  These  problems 
generally  fall  Into  two  groups:  search  and  hypothesis  generation.  (This 
division  corresponds  to  the  two  basic  psychological  models  of  probl em-sol vl ng 
and  thinking  strategies:  heuristic  search  and  hypothesis  generation  and  test, 
see  Gerw  I  n  and  Newsted,  1977.)  Search  problems  ! nterf ere  w I th  the  analyst's 
Inductive  search  for  causal  patterns  In  data  sets,  or  bias  the  search,  so  that 
certain  erroneous  or  misleading  patterns  are  easily  found  and  labeled  as  causal. 
Hypothesis  generation  problems  Interfere  with  the  analyst's  deductive  reasoning 
from  data  patterns  and  lead  the  analyst  to  generate  false  hypothetical 
explanations  from  available  data  sets.  These  false  nypotheses  may  then  mislead 
the  analyst's  subsequent  perceptions  and  analyses. 

Stech  also  noted  that  naval  analysts  themselves  seem  to  be  aware  of  the 
methodological  difficulties  of  establishing  cause  and  effect  relationships  (see, 
e.g..  Booth,  SN.1 :  470),  but  few  of  the  sampled  naval  analysts  made  any  use  of 
soph  I st I cated  methodologies  or  causal  analysis.  Some  of  these  causal 
methodologies  have  been  applied  to  problems  of  naval  Intelligence  and  others  to 
nonnava*  Intelligence  estimation  problems. 

Gfluaa.l-SQflr.sh 

The  sources  of  biases  of  causal  search  can  be  divided  Into  (1)  Incomplete 
perceptions,  (2)  mistaken  perceptions,  and  (3)  overly  simplistic  perceptions. 
Tnese  problem  areas  overlap  (Incomplete  perceptions  contribute  to  overly 
simplistic  perceptions)  and  are  more  than  what  Is  typically  metat  by 
’•‘perception".  We  use  the  'erm  "perception"  here  to  emphasize  an  inductive 
search  of  data  features  for  causal  patterns  which  then  modify  hypotneses. 
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Incomplete  perceptions.  Incomplete  perceptions  of  causal  patterns  are 
probably  largely  due  to  faulty  covariation  assessment.  That  Is,  an  analyst  may 
be  over-attentl ve  to  positive  Instances  of  covariation  of  one  event  and  another 
C++  events).  This  over-attention  to  posttlve  cases  can  suggest  a  causal  rela¬ 
tion  If  the  analyst  neglects  to  search  out  evidence  of  +-  or  -+  events  as  well. 
That  Is,  the  suspected  cause  may  be  present  when  no  effect  occurs,  or  absent 
when  the  effect  occurs.  The  analyst  should  also  confirm  the  negative  case,  that 
the  effect  Is  absent  when  the  cause  Is  absent  ( —  events).  There  Is  a  tendency, 
however,  to  not  search  beyond  the  ++  events  for  evidence  of  covariation.  In  the 
section  preceding  this  one  we  outl  Ined  a  variety  of  cross-tabulation  and  co¬ 
variation  assessment  methods  which  help  the  analyst  evaluate  a  suspected  pattern 
of  covariance. 

Mistaken,  perceptions.  Mistaken  perceptions  are  those  patterns  which  tend 
to  be  perceived  as  causal  because  of  Intuitive  cognitive  logic  or  perceptual 
bias  regarding  causal  re  I  at  I  onsh  I  ps.  For  example,  things  we  focus  our  attention 
on  tend  to  be  perceived  as  causal.  Outcomes  which  match  the  Intentions  of  an 
actor  are  naturally  assumed  to  have  been  caused  by  the  actor.  Events  which 
share  temporal  or  physical  character  I st I cs  may  be  perceived  as  causally  related. 
These  Intuitive  logical  relations  and  biases  may  lead  the  analyst  to  search  for 
data  to  confirm  these  patterns,  producing  a  belief  In  a  false  causal  relation¬ 
ship,  bolstered  by  selectively  perceived  data.  These  perceptions  and  ♦h>s  be¬ 
liefs  that  result  often  seem  Intuitively  obvious  and  amplv  confirmed  by  the  data 
which  are  selected  to  "prove"  them.  It  Is  quite  difficult.  If  not  Impossible, 
to  avoid  such  perceptions  and  beliefs  altogether  since  they  are  based  on  highly 
efficient  and  logical  concepts  about  causality.  That  Is,  for  many  or  most 
causal  relationships  these  perceptions  and  concepts  are  not  mistaken  at  al  I. 
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Rather  than  attempting  to  avoid  such  perceptions  or  concepts,  it  Is  easier  to 
attempt  to  establish  their  validity  as  systematically  as  possible. 

The  sampled  naval  analysts  typically  evaluated  causal  relationships  narra¬ 
tively.  That  Is,  evidence  in  favor  of  a  suspected  causal  relationship  was 
listed  and  evaluated  narratively  to  establish  a  coherent  relation  between  the 
suspected  cause  and  the  effect.  Evidence  for  or  against  (typical ly  against) 
other  possible  causes  would  be  listed  and  evaluated.  Rarely  was  there  any 
description  of  an  effort  mounted  to  make  these  evaluations  systematically  com¬ 
parable,  t. a.,  to  give  the  favored  and  disfavored  possible  causes  an  equivalent 
assessment,  that  is,  analysts  typically  reported  what  amounted  to  results;  the 
supported  hypothesized  causes  (and  the  dt sconf Irm ing  evidence).  The  strengths 
or  equivalence  of  the  assessment  methods  often  could  not  be  Judged. 

Causal  analysis.  Several  social  scientists  have  outlined  methodologies  for 
making  systematic  assessments  of  causal  rel ationshi ps  In  nonexper i mental 
research  (e.g..  Asher,  1976;  Blalock,  1964;  Helse,  1975).  These  methods  are 
labeled  “causal  model Ing",  or  “causal  analysis*,  and  are  techniques  for 
selecting  variables  that  are  potential  determinants  of  effects,  and  attempting 
to  Isolate  the  separate  contributions  to  effects  made  by  each  suspected  cause. 
Because  all  the  candidate  causes  are  evaluated  within  the  same  model,  the 
assessment  tends  to  be  more  systematic,  equivalent,  and  transparent.  As  we 
noted  earlier,  "causes"  can  never  be  proved  because  they  are  mental  rather  than 
physical  constructs.  Causal  models  are  merely  analytic  aids  for  evaluating  and 
assessing  hypothesized  causal  relaf lonsntps. 

Causal  models  typically  use  mathematical  regression  equations  as  causal 
equations.  That  is,  a  variety  of  suspected  causes  are  combined  in  a  linear 
equation  in  an  attempt  to  determine  the  Impact  of  each  suspected  cause  on  the 
effect.  Causes  with  nonadditive,  interactive  effects  (e.g..  multiplicative 
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ones )  can  often  be  eas  1 1  y  mod  ified  Into  additive  ones  so  that  powerf  u  i  linear 
mathematical  methods  can  be  applied  (Blalock,  1964)).  For  moderately  complex 
causal  models  a  technique  known  as  "path  analysis"  allows  for  the  estimation  of 
the  magn I tude  of  the  II nkages  between  causal  variables,  the  poss ' b I e  causal 
rel atlonsh I ps  between  variables,  and  the  direct  and  Indirect  effects  of 
variables  on  each  other.  Path  analysis  also  allows  for  tests  of  the  adequacy  of 
the  whole  causal  model  (Asher,  1976). 

The  most  sophisticated  causal  models  Include  time  as  a  major  causal 
variable  and  consider  the  possibility  of  feedbacks  In  the  system  of  causes. 
That  Is,  the  Indirect  effects  of  cause  A  on  cause  B  may  feed  back  on  Cause  A  so 
that,  at  a  later  time,  cause  A  has  a  different  effect  on  cause  B  and  on  the 
overall  effects  tn  the  system.  These  models  are  labeled  "dynamic  systems"  and 
usually  require  extensive  data  on  variables  over  time. 

O'Leary  and  Coplln  (1975,  ch.  7)  developed  a  series  of  quantitative  causal 
relationships  for  State  Department  Intelligence  analysts  tn  an  effort  to 
forecast  the  strength  of  evolving  calltlons  among  o 1 1 -export I ng  and  otl-import- 
Ing  countries.  They  were  attempting  to  translate  State  Department  analysts’ 
hypotheses  Into  quantitative  form,  so  this  application  Is  perhaps  more  relevant 
to  analyst  problems  of  hypothesis  generation  than  to  problems  of  causal  search. 
However,  because  causal  models  can  reveal  unsuspected  relations  and  Invalidate 
Intuitive  rel at,  .ns  they  also  serve  to  enhance  the  search  for  causal  patterns. 

O'Leary  ana  Coplln  developed  quantitative  Indices  of  Internal  cohesion  for 
the  two  groups  of  nations  and  of  the  bargaining  between  the  two  blocs.  Data  on 
oil  and  nonoil  transactions  (trade)  and  on  votes  on  political  Issues  were  used 
to  estimate  future  relations  between  oil-producing  states  and  oil  companies. 
These  rel atlonsh 1 ps  were  found  to  shift  over  time,  suggesting  a  dynamic  model  of 
cohesion  variables  (causes)  on  the  bargaining  variables  (effects). 
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Simplistic  perceptions.  The  last  set  of  problems  analysts  may  have  with 
causal  search  Is  related  to  overly  simplistic  perceptions.  We  use  this  term  to 
reflect  such  problems  as  "minimal  causation"  (the  tendency  to  search  for  the 
first  plausible  cause,  see  Stech  1981,  Chapter  4),  "causal  hydraulics"  (the 
tendency  to  perceive  causality  as  fixed  In  amount),  and  reduct  Ion  1st  I c  models 
(the  tendency  to  accept  too  small  a  set  of  causes  as  responsible  for  an  effect). 
These  prob I ems  may  lead  the  analyst  to  over  I ook  valid  causes,  fo  I gnore  the 
poss I b 1 1  I ty  of  multiple  causes,  or  to  see  affects  as  com  I  ng  about  In  only  one 
way. 


Search  trees;  To  heip  overcome  some  of  these  problems,  analysts  might  use 
"fault  tree"  and  "decfslon  tree"  techniques.  Fault  trees  are  diagrams  developed 
by  engineers  to  determine  how  a  particular  event  (a  fault  or  failure)  could 
occur  In  a  system.  The  engineers  reason  backwards  from  effect  fo  possible 
causes.  For  example,  the  engineers  may  want  to  determine  all  the  possible  ways 
an  automobile  might  fall  to  start.  This  technique  has  been  used  extensively  In 
nuclear  safety  ana  lysis. 

If  a  fault  tree  Is  an  effort  to  answer  the  question  "how  could  a  state  like 
this  come  about",  a  decision  tree  addresses  the  question  "given  this  state, 
where  can  events  go  from  here".  In  other  words,  a  decision  tree  might  be 
created  for  the  possible  future  actions  a  mechanic  might  take  If  confronted  with 
a  car  that  refuses  to  start.  Fault  and  decision  trees  are  examples  of  what 
psychologists  term  probieat-solvlng  search  trees. 

Tree  diagrams  serve  to  systematize  suspected  relationships  end  sequen¬ 
tial  nature  of  events  and  responses.  They  also  increase  the  poss  lb  II  ity  mat 
new  relationships  may  be  perceived  that  otherwise  might  have  been  olssea,  out 
they  do  not,  In  themselves,  ensure  that  all  possible  alternatives  ®re  included. 
Possible  casuses  way  be  omitted  from  a  fault  tree  and  possible  options  eav  5# 
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left  out  of  a  decision  tree.  There  are  no  methodological  solutions  to  this 
Incomplete  specification  of  possible  causes  or  effects,  but  some  heuristic 
approaches  may  help  the  analyst  fill  out  an  Initial  causal  search  tree. 

For  example,  the  analyst  might  explore  the  possibility  that  the  possible 
branches  of  the  cause  tree  are  limited  by  the  nature  of  the  cause-effect 
problems.  For  example,  the  analyst  may  be  able  to  establish  that  there  are  only 
three  groups  In  the  Soviet  Union  capable  of  changing  the  design  program  for 
Soviet  ships.  That  Is,  there  may  be  a  way  to  establish  that  the  tote  I  possible 
causes  of  an  effect  In  a  causal  search  tree  are  limited  to  some  finite  number. 
This  number  (cal  I  it  X)  gives  the  analyst  a  "goal  state"  —  e.g..  find  X  causes 
for  the  effect.  When  the  analyst  has  compiled  a  list  of  X  causes,  the  problem 
of  completing  the  causal  search  tree  Is  done.  This  strategy  entails  first 
exam  I n I ng  the  general  cause-effect  problem  for  the  poss I b ! I  I ty  of  a  boundary 
that  defines  and  limits  the  number  of  possible  causes,  and  then  attempting  to 
generate  all  specific  cause-effect  relations,  rather  than  the  more  typical 
reverse  approach,  in  the  reverse  approach,  the  analyst  cannot  know  If  he  or  she 
has  reached  the  stopping  point  (I.e.,  has  exhausted  ail  possible  causes). 

Many  causa-effect  profciauis,  however,  do  not  have  apparent  boundaries  around 
the  possible  branches,  and  the  tree  appears,  a  priori,  unbounced.  The  analyst's 
stopping  problem  becomes,  not  "Are  these  al  I  the  possible  causes,"  but  rather 
"Are  these  all  the  possible  causes  worth  Investigating?" 

Psychologists  studying  probl ea-soi vi ng  behavior  have  Identified  two  types 
of  Impediments  to  search  tree  construct  Ion.  One  Impediment,  *'funct  tonal 
fixedness",  ental I s  represent! ng  objects  uy  their  conventional  functions  and 
felling  to  consider  the  objects*  novel  functions.  An  analogous  block  might  be 
terms  "event  ffxedness",  the  tendency  to  consider  only  conventional  causes  for 
events  rather  tnan  novel  causes.  A  related  Impediment  resuits  from  "set 


effects",  the  tendency  to  attack  a  problem  with  one  approach  or  method  and  not 
to  change  that  approach. 

An  example  of  "set  affects"  might  be  a  mechanic,  attempting  to  determine 
the  causes  of  an  automobile's  failure  to  start,  thinking  In  terms  of  elect'- 1  cal 
problems  (dead  battery,  wet  spark  plugs,  bad  distributor),  but  not  In  terms  of 
fuel  problems  (empty  gas  tank,  blocked  fuel  line,  broken  fuel  pump).  An  example 
of  "event  f  I xedness"  might  be  the  mechan  I c  who  falls  to  real  I  ze  that  cars  may 
not  start  for  nonmechant cal  reasons  (e.g.,  attempting  to  turn  the  Ignition  key 
the  wrong  way,  attempting  to  start  while  the  automatic  transmission  Is  set  to 
"drive").  „ 

One  escape  from  functional  fixedness  Is  to  attach  specific  labels  to 
objects  and  parts  of  objects.  These  labels  allow  peop I e  to  mental  I y  connect 
objects  to  different  purposes  that  they  might  not  consider  when  presented  with 
the  objects  alone  (see  Posner,  1973;  154-158).  This  suggests  that  naval 
analysts  shou I d  change  perspect I ves  and  "d I sassemb I e"  sped  f I c  I abel s  to  the 
different  perspectives  or  the  component  parts  of  the  event.  These  labeled 
perspectives  or  parts  would  then  perhaps  suggest  more  possible  causes  for  the 
event  that  would  occur  to  the  analyst  who  only  considered  the  event  as  a  whole 
entity  from  a  single  perspective. 

This  labeling  hypothesis  suggests  that  efforts  by  naval  analysts  to  create 
event  taxonomies  which  specify  Important  features  of  events  (e.g.,  MCConnel's 
"rules  of  the  game"  effort,  Olsmukes  and  McConnell,  1979,  ch.  7)  arc  highly 
heuristic  because  the  various  features  may  suggest  new  perspectives  and  novel 
causes  to  the  analysts  building  causal  search  trees.  The  more  varied  the  ways 
analysts  are  able  to  label  or  code  an  event,  the  more  varied  the  possible  causes 
the  analysts  are  likely  to  consider  as  related  to  that  event.  Similarly, 
schemes  which  classify  the  subevents  of  an  event  (e.g.,  pre-crisis,  crlsts- 
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buildup,  height  or  crisis,  post-crisis,  return  to  status  quo)  should  facilitate 
building  causal  search  trees.  The  more  classification  schemes  the  analyst  can 
consider  (e.g.,  dimensions  of  time,  tension,  geography,  actors.  Institutions, 
technologies  are  Just  some  of  the  possible  bases  for  grouping  subevents),  the 
richer  the  set  of  causes  the  analyst  Is  likely  to  consider. 

Several  of  the  mechanisms  Ascher  (1978)  suggested  for  surpr  1  se-sens I tl ve 
forecasting  ,  e.g.,  suspension  of  plausibility  checks,  may  facilitate  expanding 
the  branches  of  the  causal  saarch  tree. 

The  phenomenon  of  "tunnel  history",  which  Fischer  (1970)  lists  as  a  problem 
with  narrative  analysis  (see  Stech,  1981,  Chapter  4,  "Narrative  Logic"),  Is  a 
set  effect.  That  Is,  explaining  naval  events  as  due  to  naval  causes  limits  the 
bul  I  ding  of  the  causal  saarch  tree.  The  I nterd I  sc  I p l I  nary  approach  taken  by 
many  of  the  sampled  naval  analysts  Is  likely  to  reduce  the  tendency  toward  set 
effects,  but  analysts  might  enhance  this  positive  effect  by  dlrec+iy  Involving 
analysts  from  different  disciplines  In  the  causal  troe  construction  and 
evaluation  process. 

Tree  problems.  Psychological  research  on  fault  trees  (FIschhcff,  Slovtc, 
and  Lichtenstein,  1978)  shows  that  analysts  will  not  typically  notice  the 
omission  of  Important  causal  branches.  For  example,  Flschhoff  and  his 
colleagues  showed  people  (including  au?o  mechanics)  various  versions  of  a  fault 
tree  for  the  problem  of  the  nonstarting  car,  and  asked  for  estimates  of  the 
proportion  of  no-starts  caused  by  the  category  "all  other  problems".  As  various 
causal  branch.es  (#.g.,  fuel  system)  are  deleted  from  the  tree,  the  proportion  of 
non-starts  attributed  to  "all  other  problems"  should  increase.  It  did  not. 
People  seemed  not  to  miss  the  absence  of  important  causal  branches,  and  seemed 
unable  to  appreciate  how  many  causes  had  been  omitted  ♦rom  the  pruned  fault 
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trees.  In  other  words,  there  was  a  faulty  tendency  to  overestimate  the 
completeness  of  Incomplete  causal  diagrams.  Neither  self-rated  knowledge  nor 
years  of  mechanclal  experience  Improved  the  ability  to  detect  this 
Incompleteness. 

FIschhoff  and  his  colleagues  found  somewhat  greater  sensitivity  to 
Incomplete  fault  trees  when  people  were  explicitly  urged  to  consider  the  tree's 
completeness  and  think  about  possible  causes  that  might  be  left  out.  In  this 
case  people  were  more  I Ikely  to  estimate  that  causes  were  missing,  but  even 
these  estimates  tended  to  be  low,  I.e.,  there  were  more  causes  missing  than  even 
this  sensitized  group  estimated. 

These  results  suggest  that  analysts'  efforts  to  fill  out  and  el aboraio  a 
causal  search  tree  are  probably  well-spent  and  will  tend  to  pay  off  In  possible 
causes  that  woul  d  otherw  i  se  be  overt  ooked.  The  I  ssue  stl  l  I  open,  however,  I  $ 
how  the  analyst  can  determine  the  point  to  stop  searching  for  causes. 

Steep f f\g  problems.  While  efforts  to  overcome  Impediments  to  causal  search 
tree  constructs  will  reduce  the  chances  that  naval  analysts  will  overlook 
Important  causes,  they  do  not  solve  the  stopping  problems  —  knowing  when  ♦ne 
causal  tree  Is  sufficiently  completed.  Tnls  Is  essentially  a  problem  of 
induction  and  Information  Integration:  given  a  rich  causal  tree,  does  the 
evidence  suggest  that  some  subset  of  the  possible  causal  branches  Is  an  adequate 
or  satisfactory  explanation  of  the  event. 

Two  methods  which  help  analysts  with  this  stopping  problem  were  noted  above 
under  "Weighting  Data":  Bayesian  techniques  and  linear  modeling.  Linear  models 
are  the  basis  for  most  causal  modeling  approaches  and  may  be  the  most  compatible 
method  for  assessing  whether  the  cause  tree  seems  sufficiently  complete.  The 
statistics  of  causal  analysis  techniques  allow  the  analyst  to  estimate 
unexplained  variance,  which  corresponds  to  tn®  role  of  undetermined  causes.  If 
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unexplained  variance  Is  too  high  (an  analytic  judgment  must  be  made  as  to  what 
"too  high”  means),  the  causal  tree  Is  Incomplete  and  the  causal  search  should 
continue.  The  use  of  Bayesian  techniques  al lows  the  analyst  to  estimate  the 
I Ikel Ihood  of  an  event  given  various  causes,  and  to  estimate  the  subjective 
probab 1 1  I ty  that  any  causal  explanation  of  the  event  I  s  true.  I  f  the  anal  yst 
finds  his  or  her  subjective  probabilities  for  the  various  causal  explanations 
are  all  too  I  ow  (agal  n  "too  I  ow"  requ  I  res  an  ana  I  yt  I  c  Judgment),  the  causal 
search  I s  I ncomp I ete. 

One  of  the  more  wide-spread  uses  of  Bayesian  techniques  in  causal  analysis 
ta-os  place  In  medical  diagnosis  (see  e.g„  Gorry,  et.  al.,  1973);  Lusted,  1968; 
SicNell  et,  al.,  1975;  and  Schwartz,  et.  al.,  1973).  Decision  aided  medical 
dtagnr'ds  makes  use  of  decision  trees  which  specify  the  possible  actions  and 
tests  th«  physician  can  fake  and  their  possible  consequences  for  a  patient  with 
one  of  several  possible  maladies.  Bayesian  techniques  are  used  to  evaluate  the 
physicians  subjective  probability  estimates  that,  given  certain  test  results 
and  symptons,  the  patlent»s  malady  Is  caused  by  one  disease  rather  than  another. 
As  further  Information  on  tests  or  patient  responses  to  treatment  Is  acquired 
(I. a.,  as  the  physician  and  patient  move  to  different  nodes  In  the  decision 
tree)  8ayes  formula  can  be  used  to  update  the  physician's  estimates  of  the 
probabilities  of  various  causes.  Bayesian  techniques  are  especially  useful  In 
compensating  for  the  tendencies  to  overemphas I ze  case  data  (e.g.,  a  positive 
test  result)  and  to  underweight  base-rate  data  (e.g.,  the  Incidence  of  the 
disease  In  the  population  at  large). 

Bayesian  techniques  do  not  handle  effectively  situations  where  multiple 
causes  are  operating,  e.g.,  a  pat'ent  with  two  disorders  of  overlapping  nature. 
However,  It  Is  for  Just  such  situations  that  causal  model  Ing  was  developed. 
Similarly,  Bayesian  techniques  may  be  less  effective  In  cases  where  causes 
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change  over  time,  Oynamlc  systems  modeling,  however.  Is  a  means  for  attacking 
this  problem.  This  suggests  that  the  selection  of  a  particular  analytic  aid  for 
a  causal  analysis  problem  Is  likely  to  be  a  heuristic  or  experimental  process. 
The  "right"  aid  may  not  be  the  first  one  selected.  An  experimental  approach  to 
aids,  I. a.,  a  willingness  to  try  various  types  and  combinations  may  be  required 
(Krlscher,  1980). 

Hy.o.olhes.1  £-Gener  at  1  on 

Several  biases  exist  In  the  process  of  causal  hypothesis  generation,  t.e., 
the  tendency  to  generate  some  types  of  causal  explanations  and  not  others.  We 
do  note  the  ease  with  which  people  (Including  scientists  using  sophisticated 
cause  assessing  methods)  could  detect  a  "presumptive  agency"  connecting  a  sus¬ 
pected  cause  with  an  effect  and  find  confirming  evidence  fer  this  hypothesis, 
even  when  It  was  false. 

In  contrast  to  the  search  approach  to  causal  analysis,  w.ilch  emphasized 
recognition  and  organization  of  pattern  features  In  data  sets,  the  hypothesis 
generation  approach  focuses  on  the  logical  generation  of  hypotheses,  their  test, 
and  subsequent  revision.  Hypothesis  generation  problems  then  are  likely  to 
occur  at  either  the  generation,  test,  or  revision  phases.  W«  will  discuss  the 
latter  two  problem  areas  later,  when  we  consider  theories.  In  the  present 
section  we  consider  hypothesis  generation  problems. 

The  biases  In  causal  hypothesis  generation  tend  to  fal  I  Into  two  groups: 
those  that  are  largely  due  to  cognitive  processes  and  those  that  appear  to  be 
based  on  assumptions  about  causality  In  soc  I  a  I  rel  atlonsh  I  ps.  The  cognitive 
problems  seem  to  occur  because  certain  features  of  the  Information  stored  In 
memory  about  effects  and  possible  causes  tend  to  suggest  certain  causal  explana¬ 
tions  (which  may  be  Inappropriate).  When  these  data  features  are  largely  social 
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(e.g.,  a  liked  actor  versus  a  dlsltked  actor),  the  hypothesis  generation  problem 
can  be  labeled  social,  although  It  may  have,  at  a  more  abstract  level,  a  cogni¬ 
tive  bas!  s. 

Cognitive  problems.  Three  types  of  bias  which  may  lead  the  analyst  to 
generate  faulty  causal  hypotheses:  fundamental  attribution  error,  representa¬ 
tiveness,  and  determinism. 

The  fundamental  attribution  error  Is  the  tendency  to  attribute  behavior  to 
correspond  I ng  personal  dispositions  of  the  actor  and  not  to  environmental 
causes.  In  large  part  this  tendency  Is  due  to  focusing  of  attention  and  to  the 
representativeness  heuristic.  The  focusing  of  attention  on  an  object  (or  actor) 
Increases  the  likelihood  that  the  object  of  attention  will  be  perceived  as 
causing  events.  The  representativeness  heuristic  refers  to  the  tendency  to  Iook 
for  causes  whose  principal  features  match  those  of  the  effect.  Oetermlntsm  is 
the  tendency  to  seek  nonprobab 1 1 1  Stic  causes  for  phenomena  and  to  not  consider 
the  possibility  of  causal  forces  that  have  probabilistic  effects. 

An  actor  Is  typically  the  center  of  attention,  and  thus  Is  seen  as  causal, 
and  the  causes  of  the  effects  produced  by  the  actor  are  typically  deemed  to  have 
originated  with  the  disposition  of  the  actor  rather  than  with  the  pressures  of 
the  environmental  background. 

This  distinction  between  dispositional  (or  Internal)  sources  of  behavior 
and  environmental  (or  external)  sources  Is  one  of  considerable  Importance  for 
naval  and  mill tary  anal ysts.  Old  the  Sov I ets  become  I nvol ved  tnt  his  crisis 
because  of  their  overall  plan  to  destabilize  the  region  (Internal  cause)  or 
because  their  client  state  ts  threatened  by  a  Western  client  (external  cause). 
Old  the  Soviet  naval  construction  plan  change  In  response  to  Soviet  geopolitical 
strategy  (Internal)  or  In  response  to  naval  threats  from  their  adversaries 
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(external).  To  what  extent  are  Soviet  actions  due  to  a  mixture  of  Internal  and 
external  causes? 

Social  scientists  have  developed  an  extensive  and  elaborate  set  of  quanti¬ 
tative  methods  to  deal  with  one  form  of  this  causal  problem;  determining  the 
causes  of  arms  races.  These  methods  could  be  extended  to  efforts  to  model 
crisis  Interactions,  diplomatic  penetrations,  aid  for  conflicting  client  states, 
etc.  In  our  sample  of  naval  estimates  none  of  the  various  dynamic  arms  models 
were  used. 

It  Is  somewhat  surprising  for  two  reasons  that  none  of  the  naval  analysts 
sampled  used  quantitative  arms  race  models  In  their  analyses.  First,  the  naval 
analysts  themselves  (e.g.,  Bowen,  NP$P.  ch.  4)  occasionally  drew  parallels 
between  previous  naval  races  and  the  Sov I et-Amer I  can  naval  rivalry.  cor 
example,  Bowen  (p.  57)  describes  the  cr I cumstances  of  the  current  Sovlet- 
Amertcan  naval  balance  as 

...similar  to  those  that  preva I  led  at  the  beginning  of  the 
century  when  Germany  challenged  the  supremacy  of  Britain's 
navy. 

Second,  many  arms  rivalries  In  the  19th  and  20th  centuries  have  been  naval. 
Huntington  (1958)  listed  thirteen  arms  races  In  this  period,  of  which  seven  and 
a  third  were  naval  (the  third  of  a  case  Is  the  nuclear  competition  of  the  United 
States  and  USSR).  While  Hunt  I  ngtons  list  may  not  be  exhaust  I  ve.  It  Is  quite 
likely  that  the  high  proportion  of  naval  <*aces  would  be  found  In  a  more  complete 
1 1  st.  Certal  n  I  y,  one  cou  I  d  cone  I  uce  that  about  half  of  the  major  arms  races, 
tor  which  social  scientists  have  developed  quanMt?*i,s  analytic  models,  have 
been  naval. 

Two  main  advantages  of  arms  race  models  are  that  they  (1)  make  more  expli¬ 
cit  and  mathematically  precise  the  analyst's  Implicit  assumptions  and  Intuitive 
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hypotheses  about  arms  competition,  and  (2)  provide  a  clear  distinction  between 
foreign-induced  and  self-induced  forces  In  a  nation's  arms  program  (Gantzel, 
1973;  Wallace  and  Wilson,  1978).  Other  aspects  of  the  nature  of  arms  competi¬ 
tion  can  be  Included  In  these  models,  e.g.,  whether  the  states  Involved  are 
competing  In  numbers  or  technology  or  both  (e.g..  Hoi  I  1st,  1977;  Huntington, 
1958;  tuterbacher,  1976).  It  is  also  possible  to  model  an  arms  competition 
between  two  nations  (e.g.,  the  United  States  and  the  USSR)  at  the  same  time  that 
competition  between  pacts  and  alliances  are  examined  (e.g.,  NATO  and  Warsaw 
Pact,  see,  Rattinger,  1973;  Wallace  and  Wilson,  1978).  That  Is,  there  may  be 
all  lance  or  pact  causal  factors,  as  well  as  I ntranat tonal  and  international 
causes  for  arms  competition.  Arms  models  also  help  the  analyst  avoid  "mirror 
Imaging".  That  Is,  while  one  state  may  be  reacting  largely  to  Its  competitor's 
behavior,  the  competitor  state  may  be  responding  primarily  to  Internal  forces. 
"Mirror  Imaging"  Is  the  bias  toward  perceiving  such  situations  as  symmetrical, 
l.e,,  both  competing  states  are  reacting  to  the  same  kinds  of  factors.  Several 
arms  race  modelers  (e.g.,  Hoi  I  1st,  1977;  tuterbacher,  1976;  Rattinger,  1975; 
Wallace  and  Wilson,  1978)  using  different  models  and  various  data  sets  a* I 
assent  ally  concluded  that  while  the  USSR's  arms  and  especially  Its  strategic 
mlssl'e  programs  were  reactive  to  Western  (especially  United  States)  arms,  the 
United  States'  programs  were  not  symmetrical;  U.S.  programs  showed  more  reac¬ 
tion  to  Internal  forces  (cost,  tecnology,  previous  arms  spending)  than  did  the 
Soviet  programs.  It  Is  a  mistake  to  dismiss  arms  race  models  (as  does  Wohls- 
tetter,  1975;  47)  simply  because  the  two  nations  Involved  do  not  behave. In 
exactly  the  same  (or  a  symmetrical)  way.  In  fact,  tt  Is  because  the  nations  may 
not  he  reacting  Identically,  or  responding  to  forces  which  are  the  mtrror  Image 
of  each  other,  that  arms  race  models  are  particularly  helpful. 
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Explicit  models  of  arms  competition  may  yield  some  useful  predictive  Indi¬ 
cators  as  wel  I  as  systematize  rhe  analyst's  reasoning  on  causal  factors.  For 
example,  Wal  lace  (1979)  found  that  the  product  of  the  smoothed  »-ate  of  arms 
Increases  for  pairs  of  disputatious  nations  predicted  whether  war  would  follow 
the  arms  competition.  Rapidly  accelerated  arms  races  escalated  to  war  In  23  out 
of  28  cases,  while  disputes  not  preceded  by  accelerating  arms  competition  re¬ 
sulted  In  war  only  3  times  out  of  71  cases. 

An  excellent  example  of  quantitative  arms  race  modeling  combined  with 
detailed  qualitative  descriptive  analysis  Is  Lambelet's  three-part  series  on  the 
Anglo-German  (Dreadnought  race  (1974,  1975,  1976).  Lambelet  (1976:  50)  presented 
a  diagram  (Figure  ^ 4 )  Illustrating  how  his  traditional  analysis  and  his 
numerical  methods  were  Integrated  In  his  study.  His  approach  reinforces  the 
po1  nt  made  here  that  quantitative  assessments  of  causal  factors  can  be  a  useful 
tool  for  determining  tne  causes  of  a  ration's  naval  objectives. 

Social  problems.  Two  tvoes  of  causal  problems  can  be  related  to  the 
tendency  to  generate  causal  hypotheses  on  the  basis  of  social  Intuitions.  One 
of  these  problems  was  the  tendency  to  attribute  motivational  causes  when  the 
consequences  of  actions  are  more  foreseeable.  Actors  who  are  perceived  as  able 
to  foresee  *he  consequences  of  their  acts  were  also  perceived  as  more  motivated 
(rather  than  pressured  by  external  forces)  to  bring  those  consequences  about. 

A  second  problem  of  social  perception  of  causes  Is  the  tendency  to  attri¬ 
bute  the  "good*  actions  (those  approved  of  by  the  observer)  of  I  Iked  actors  to 
dl sposl t Ions  and  of  disliked  actors  to  luck,  chance,  or  situation,  on  the  one 
hand,  and  to  attribute  "bad*  actions  of  liked  actors  to  luck,  chance,  or  situa¬ 
tion,  and  of  disliked  actors  to  dispositions,  Olsltked  actors.  In  other  words, 
tend  only  to  be  seen  as  planning  bad  actions  and  >?s  being  forced  (or  stumbling 
accidently)  Into  good  actions. 


The  methods  outlined  above  In  this  section  for  systematizing  causal 
hypotheses  could  assist  the  analyst  to  avoid  these  social  cause  biases.  Another 
technique  was  outlined  In  Stech,  1981:  Kelly's  consensus,  consistency,  and 
distinctiveness  method.  This  method  can  be  viewed  as  a  special  form  of  causal 
model  aimed  at  the  statistical  features  of  dispositional  and  situational  cause? 
of  actions. 


F.  PREDICTIONS 

Forecasting  would  be  an  absurd  enterprise, 
were  It  not  Inevitable. 

Bertrand  de  Jouvenel , 

Jha.  Act  at  Canjactur.s 

Naval  analysts  seem  to  share  many  ot  the  same  problems  experienced  by  other 
forecasters.  The  relative  lack  of  specific  predictions  by  naval  analysts  made 
It  Impossible  In  this  study  to  systematically  assess  the  prediction  or  forecast¬ 
ing  track  records  of  naval  analysts.  Such  assessments  are  an  excellent  means 
for  prov I d I ng  the  analyst  with  se I f -correct I ng  feedback.  Th  1  s  suggests  that 
naval  analysts  could  Improve  their  prediction  and  forecasting  efforts  by  (I) 
frequently  making  specific  end  precise  predictions,  (2)  regularly  comparing 
predictions  to  outcomes  and  assessing  the  frequency,  magnitude,  and  direction  of 
errors,  and  (3)  using  this  track  record  feedback  to  modify  their 
prediction/forecasting  methods. 

Because  analysts  now  tend  to  make  vague,  Qelphlc  forecasts  or  Aesopian 
estimates.  It  Is  difficult  or  Impossible  to  gauge  the  analyst's  accuracy.  “  s 
leoves  snaysts  with  a  very  subjective  Impression  of  their  estimation  track 
record  end  the  possibility  that  analysts,  with  the  advantages  of  hindsight,  will 
perceive  their  past  track  records  as  more  precise  an\  predictive  than  they 
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actually  were.  A  subjective,  retrospect  I  v<*  approach  to  self-appraisal  is 
unlikely  to  yield  the  detailed  feedback  Information  on  Inaccuracy  that  analysts 
could  use  to  systematically  Improve  tholr  methods. 

Identifying  Assumptions 

A  main  problem  preventing  accuracy  In  forecasts  and  predictions  Is  the 
central  role  of  rh«  forecaster’s  assumptions  (Ascher,  1978).  A  careful  rnd 
systematic  prospective  assessment  o'  the  prediction  track  record  can  assist  the 
analyst  to  refine  assumptions. 

aacKcaslInq 

A  related  strategy  Is  to  employ  frequent  "backcas+I ng  exercises"  to  deter¬ 
mine  whether  the  assumptions  underlying  tne  forecast  are  true  for  past  and 
present  data  (Ascher,  1978;  Morgenstern,  Knorr,  and  Hetss,  1973).  As  Ascher 
noted  (p.  8)  forecasters  often  fall  to  examine  the  biases  of  their  predecessors 
and  neglect  to  adjust  their  forecasts  In  a  direction  that  would  rectify  earlier 
forecasts.  He  wrote  (p.  110): 

...the  use  of  prev  I  ous-error  feedback  Is  lacking  In  the 
forecasting  efforts  In  every  area  except  that  of  certain 
short- term  econooetr I c  forecast  I ng  mode  I s . 

This  nonuse  of  error-correction  feedback  seems  due  to  the  forecasters’  beliefs 
that  their  assumptions  about  the  future  already  Incorporate  all  the  data  per¬ 
tinent  to  known  trends.  In  many  cases,  however,  the  forecasters’  methods  cannot 
accurately  predict  known  trends,  lei  alone  the  unknown  future. 

The  fal I ure  of  Intel  I  I gence  analysts  to  backcast  has  been  noted  sever  al 
times  In  connection  with  the  underestimation  of  Soviet  strategic  missile 
Inventories  (Sul  I  Ivan,  In  Godson,  1980}  Wohlstetter,  1975a),  i.o.,  estimators 
continued  to  underestimate  capabilities  even  after  evidence  of  past  underestl- 


::3 


mates  were  noted,  Including  a  warning  from  Soviet  leader  Brezhnev  that  the  West 
was  under counting  I  Among  the  explanations  fo  this  persistent  underestimation  of 
capabilities  Is  the  hypothesis  that  U.S.  Intelligence  analysts  had  mlspercelved 
Soviet  strategic  Intentions  and  requirements,  and  had  "m I rror- Imaged"  the  Soviet 
Intentions  to  correspond  to  our  own  (e.g.,  see  Sullivan,  In  Godson,  1 980:  62). 

Bootstrap q I na 

A  second  technique  that  can  help  analysts  to  clarify  their  assumptions  Is 
bootstrapping.  That  Is,  a  quantitative  model  of  the  analyst’s  predictions  Is 
constructed  using  the  data  Inputs  considered  by  the  analyst  and  fitting  these 
variables  to  the  analyst's  predictions.  This  model  of  the  analyst  will  refU/t 
the  data  variables  that  most  Influenced  the  predictions,  thus  providing  me 
analyst  with  quantitative  Information  on  the  Ingredients  of  his  or  her  predic¬ 
tions.  By  knowing  which  variables  most  heavily  Influence  his  predictions  the 
analyst  can  better  assess  his  prediction  assumptions. 

Two  of  the  quantitative  studies  done  by  O’Leary  and  Coplln  (1975)  for  State 
Department  Intel  I  tgence  analysts  used  a  quas I -bootstrapp I ng  approach  toward 
Improving  predictive  capability  (although  It  was  not  Identified  as  bootstrapping 
by  the  authors).  In  one  effort  ("Predicting  political  Instability  In  tropical 
Afrtca",  ch.  2,  O’Leary  and  Coplln,  1975)  they  evaluated  the  views  of  State 
Department  analysts  on  the  causal  variables  leading  to  political  Instability  In 
tropical  African  nations.  From  the  analysts'  narrative  analyses,  O’Leary  and 
Coplln  abstracted  a  variety  of  hypothesized  causal  factors.  These  factors  were 
then  quantl fled  and  regressed  on  measures  of  pol I f leal  Instabl I Ity.  Had  the 
study  been  a  truly  bootstrapping  effort,  the  variables  would  be  regressed  on 
analysts'  predictions  of  political  Instab! I Ity,  y lei  ding  a  model  of  the  ana¬ 
lysts'  predicting*.  The  O'Leery  and  Coplln  approach  was  to  develop  a  model  of 
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the  situation,  but  the  same  techniques  could  be  used  In  a  bootstrapping  effort. 
Both  model  building  techniques  aid  the  analysts  by  explicitly  relating  variables 
to  predictions,  thus  clarifying  assumptions.  O’Leary  and  Copl in  (ch.  6)  fol¬ 
lowed  a  similar  approach  In  their  effort  to  develop  a  Quantitative  model  to 
predict  violence  In  the  Middle  East.  In  this  case  the  State  Department  ana¬ 
lysts'  projections  (I.e.,  forecasts  and  predictions  of  Middle  East  Violence) 
were  the  predicted  variables,  and  O'Leary  and  Copl In  attempted  to  mathematically 
relate  event  data  set  measures  to  the  analysts'  projections.  The  analysts' 
projections  of  violence  were  closely  related  to  the  weekly  and  monthly  fre¬ 
quencies  of  violent  acts  In  the  area.  In  effect,  the  analysts'  projections  of 
violence  in  the  future  were  strongly  Influenced  by  the  frequency  of  violence  In 
the  past  week  and  month. 

O'Leary  and  Coplln'j  analysis  Included  an  outline  of  an  assessment 
technique  to  test  the  validity  of  the  analysts'  assumptions  as  revealed  by  the 
quantitative  model,  which  seems  to  be  one  of  the  more  useful  consequences  of 
bootstrapping  the  analysts'  predictions. 

Ssnsll.LY.1  tv.  Testing 

Other  problems  include  the  tendencies  to  overweight  case  data  and 
underweight  base-rate  data  In  predictions,  and  the  tendency  to  overestimate  the 
predtcttve  validity  of  Indicator  variables.  A  partial  antidote  to  these 
tendencies  Is  sensttivlty  testing,  that  Is,  asking  the  analyst  to  consider  how 
her  or  hts  predictions  would  be  different  if  the  base  rates  were  vastly 
different,  or  if  the  predictive  validity  of  the  Indicators  were  varied.  These 
sensitivity  exercises  tend  to  make  predictions  more  regressive  (I.e.,  closer  to 
the  normative  statistical  predictions,  see  FIschhoff,  Slovic,  and  Lichtenstein, 
1979).  Sensltivtty  tests  seem  to  lead  to  greater  attention  to  the  predictive 
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and  diagnostic  power  of  the  variables  used,  and  predictions  more  in  line  with 
this  awareness. 

Validity  of  Future  Assumptions 

The  matn  conclusion  of  Ascher's  (1978)  and  Morganstern,  Knorr,  and  Helss' 
(1973)  evaluations  of  forecasting  Is  that  forecasting  accuracy  Is  dependent  on 
the  val  I d t ty  Xq  the  future  of  the  centra!  assumpt I ons  made  by  the  f orecaster. 
The  variables  selected  by  the  forecaster  may  be  adequate  to  explain  past  events, 
but  the  future  structure  of  the  problem  may  change,  making  these  variables  less 
Important.  The  techniques  described  above,  assessing  the  prediction  track 
record,  backcastlng,  bootstrapping,  and  sensitivity  testing,  hel p  the  analyst 
deter m  t  ne  his  or  her  assumptions,  and  assess  the  I r  val I d I ty  I n  the  past,  but 
they  do  I Ittle  to  aid  the  analyst  to  assess  the  val idity  In  those  assumptions  in 
the  future.  To  the  degree  that  future  trends  tend  to  reflect  past  trends,  these 
techniques  help  the  analyst  to  make  systematic  projections.  But  If  the  future 
Is  unlike  the  past  and  full  of  major  surprl  ses,  upheaval  s,  and  revol  utlonary 
event;  the  past,  and  methods  which  help  the  forecasting  analyst  make  systematic 
use  of  the  past.  Is  of  much  less  value. 

Simple  Model s_an.d  Surprising  Futures 

The  widespread  finding  that  relatively  stmpie  actuarial  and  econometric 
models  outpredtct  the  forecasts  of  human  estimators  in  the  short  and  medium  +erm 
suggests  that  analysts  should,  as  a  minimum,  develop  a  simple,  quantitative 
model  of  the  phenomenon  they  are  attempting  to  predict.  This  simple  model 
should  be  val  I  dated  on  past  ev  i  dence  and  used  as  a  basel  Ine  agal  nst  wh  Ich  the 
analyst  can  attempt  to  refine  and  fff  I  ne-tune"  esti  mates.  The  basel  Ine  model 
would  serve  to  highlight  those  particular  facts  and  reasons  the  analyst  believes 
the  futures  will  not  be  a  simple  extrapolation  from  the  past.  By  serving  as  a 
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val  i  dated  representation  of  the  “surprise  free'*  future,  the  simple  model.  In 
effect,  enables  the  analyst  to  concentrate  on  those  forces  and  developments  that 
may  cause  surprises.  To  the  degree  that  the  future  j_s  like  the  past,  the  simple 
model  will  do  the  best  job  of  extra po  I  at  I  ng  the  I  mportant  variables.  But  the 
model  lacks  the  analysts'  Insights,  experience,  and  Intuitions  about  new  trends, 
incipient  developments,  sudden  variations,  changes  In  context  or  tone.  These 
subtle  cues  can  only  be  appreciated  by  the  analyst,  and  the  use  of  a  basel I ne, 
"surpr t se-f ree"  model  may  allow  the  analyst  the  opportunity  to  pursue  these 
leads  to  the  surprising  future, 

A  Science  Analogy 

A  useful  analogy  can  be  drawn  between  the  situation  described  In  the 
preceding  section  and  Kuhn's  (1970)  sociological  description  of  “normal"  science 
and  "revolutionary"  or  "paradigm  shift"''  science.  Normal  science  Is  the  use  of 
proved  and  accepted  scientific  methods  to  solve  problems  and  puzzles  which  bear 
Importantly  on  a  scientific  field.  Problems  are  attacked  that  are  widely  be- 
I  loved  to  be  solvable  and,  once  solved,  become  Important  parts  of  a  major 
scientific  theory.  Kuhn  has  much  more  to  say  about  normal  science,  but  the 
point  here  is  that  it  is  analogous  to  the  development  and  use  of  simple  quanti¬ 
tative  models  to  deal  with  the  "surpr I se-f ree"  aspects  of  prediction.  Such 
models  provide  a  means  to  make  accurate  extrapolations  from  past  evidence  Into 
the  future  using  agreed-upon  methods  and  data  to  solve  Important  prediction 
problems. 

A  scientific  revolution  occurs  when  a  major  scientific  theory  is  changed, 
t.e.,  the  scientific  paradigm  shifts.  This  means  that  the  I  nterpretations  of 
relationships  observed  In  the  past  are  changed,  viewed  from  a  completely  new 
perspective,  and  given  a  different  meaning.  Such  shifts  come  about,  Kuhn 
believes,  because,  In  the  course  of  normal  science,  new,  surprising  and 
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unexpected  phenomena  are  constantly  uncovered.  Such  anomalies,  as  Kuhn  labeled 
them,  are  unexplainable  or  even  un Interpretable  within  the  context  of  the  extant 
paradigm.  For  most  scientists  they  are  not  Important  problems  because  they  fall 
outside  most  of  the  categories  and  classification  schemes  of  the  paradigm. 
Either  the  anomaly  seems  unsolvable  within  the  methodological  or  theoretical 
context  of  the  paradigm,  or,  no  I  nterpretab  I  e  solution  seems  possible,  or  the 
anomaly  Is  Interpreted  as  part  of  the  paradigm  after  ail. 

Kuhn  argued  that  anomalies  tend  to  accumulate,  however,  and  he  wrote  (1970: 

52). 


...the  scientific  enterprise  has  developed  a  uniquely 
powerful  technique  for  producing  surprises  of  this  sort. 

The  buildup  of  surprises  and  anomalies  leads  to  a  blurring  of  the  paradigm  and  a 
loosening  of  the  Informal  rules  for  normal  research. 

Eventually,  normal  science  experiences  a  crisis,  and  a  fundamental  shift  of 
perception  and  belief  occurs  as  scientists  accept  a  new  theory  to  Interpret  both 
the  old  paradigm  and  the  accumu.ated  anomalies.  This  new  theory,  developed  In 
large  part  from  the  effort  to  explain  the  surprises  generated  from  normal 
science,  reorganizes  how  the  oid  data  and  the  new  anomalies  are  perceived  and 
organized.  Scientific  theory  Is  transformed.  Many  features  of  the  old  Inter¬ 
pretation  remain,  but  the  enttre  situation  receives  a  new  Interpretation. 

The  use  of  simple  quantitative  prediction  models  In  naval  analysis  Is 
likely  to  "solve'*  many  normal  prediction  problems  as  wol  I  as  generate  many 
anomalies  and  surprises.  The  naval  analyst  should  be  especially  concerned  with 
these  unusual  d  I  scover  I  es  and  should  attempt  to  ref  or  mu  I  ate  the  concepts  and 
categories  used  to  explain  the  situation  so  as  to  Incorporate  the  anomalies. 
These  efforts  may  lead  to  a  crisis  between  the  new  Interpretation  and  the  simple 
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models,  which  may  lead  to  a  revolution  and  paradigm  shift,  I.e.,  a  new 
theoretical  view  of  Soviet  naval  Intentions.  If  this  new  theory  Is  successful 
I n  accounting  for  the  anomal  les,  It  can  be  the  basis  for  a  new  set  of  simple 
quantitative  models  (although  these  will  be  significantly  different  from  the 
succeeded  models).  The  new  models  will,  like  the  old,  generate  both  solutions 
and  new  anomal les. 

The  predictive  advantages  of  using  a  normal  sclence/sl mpl e  model  and  re¬ 
volutionary  scientist/analytic  speculation  system  Is  that  the  analyst  Is  foc¬ 
using  on  surprises,  anomalies,  and  the  unexpected,  but  Is  "backed-up"  by  a 
rel  table  theoretical  base.  The  normal  theoretical  base  of  the  s I mp I e  model s 
will  cope  with  "surpr I se-free"  situations,  freeing  the  analysts  to  develop  new 
frameworks  to  Incorporate  these  elements  of  the  present  lanomolles)  that  betoken 
the  surprises  of  the  future.  Many  anomalies  will  have  no  scientific  relevance, 
they  will  be  data  collection  errors,  deceptions,  accidents,  noise  In  the  analy¬ 
tic  channels,  etc.  Distinguishing  between  the  meaningless  anomalies  and  those 
that  signal  the  trends  of  the  future  Is  a  major  task.  Th  >  use  of  simple  model¬ 
ing  of  normal  theories  ana  a  "surpr I se-f ree"  future  might  free  the  analyst  to 
create  the  new  theories  and  consider  the  uncertainties  necessary  to  anticipate  a 
surprising  future. 

G.  TWEQR I ES 

Life  Is  the  art  of  drawing  sufficient  conclusions  from 

Insufficient  premises. 

Samuel  Butler,  Notebooks 

There  Is  nothing  like  a  theory  for  blinding  the  wise. 

George  Merl dlth, 

The  Ordeal  of  Richard  Feverel 
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There  is  substantial  evidence  that  people  (Including  scientists)  are  ex¬ 
tremely  reluctant  to  give  up  a  useful  theory.  This  reluctance  extends  to  situa¬ 
tions  In  which  considerable  evidence  discrediting  the  theory  ts  available,  and 
theories  may  survive  even  a  complete  disconf trmatiun  of  their  evi dene i ary  bases. 
This  seems  to  be  due  to  several  factors.  Theories  serve  functions  other  than 
accurate  prediction.  People  are  reluctant  to  dismiss  a  useful  theory  on  the 
basi  s  of  evi  dence  that  mi  ght  be  erroneous.  Negative  evidence  is  d  I  f  f  1  cu  1 1  to 
Integrate  Into  a  theory.  Multiple  hypothesis  testing  is  extremely  difficult  to 
conduct.  People  tend  to  seek  and  accept  confirming  evidence  more  readily  than 
refuting  evidence. 

Despite  such  difficulties  a  few  of  the  sampled  naval  analysts  did  employ 
multiple  hypothesis  testing  or  disconf irmaticn  strategies,  and  several  analysts 
seemed  aiert  to  the  significant  of  negative  evidence. 

If  there  was  one  main  characteristic  problem  of  theorizing  among  the 
sampler  naval  analysts  It  was  perhaps  the  reluctance  to  make  specific 
predictions  on  the  basts  of  various  theories  and  to  systematically  and 
repeatedly  test  theories  against  one  another.  In  particular,  there  was  too 
I  tttle  expl fett  d t sconf I rmat Ion  and  too  much  confirmation.  There  was  very 
1 1 ttl e  effort  devoted  to  dev el  oping  methods  of  test  I ng  theories  w  h I ch  woul d 
reflect  quantitatively  the  degree  to  which  evidence  supports  or  does  not  support 
any  hypothesis.  Although  we  found  naval  analysts  updated  their  theories,  the 
lack  of  explicit  methods  for  evaluating  the  degree  of  support  that  evidence 
convoys  on  a  given  hypotheses  made  It  difficult  to  ascertain  exactly  why  the 
analyst  changed  the  theory. 

A I  though  there  are  aids  aval  table  to  analysts  which  quantify  hy  pothes  I  s 
testing  (e.g„  Bayes  Theorem  Is  specifically  formulated  to  update  a  prior  hypo¬ 
thesis  given  new  information),  there  are  no  quantitative  aids  that  prescribe  how 
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or  when  scientists  should  reformulate  theories.  As  Kuhn  (1970)  describes  the 
structure  of  scientific  revolutions,  the  process  Is  largely  a  social  one  of 
scientists  arguing,  replicating,  confirming,  revising,  and  generally  shaping  and 
Influencing  each  other’s  views  and  research.  Scientific  discoveries  which  fall 
to  generate  Interest  in  other  scientists  die  on  the  vine  unless  rediscovered  In 
the  wake  of  a  scientific  crisis  and  revolution.  Similarly,  an  Intelligence  or 
naval  analyst  may  reformulate  a  theory  of  Intentions  that  produces  better  esti¬ 
mates,  but  unless  the  new  theory  Is  communicated  to  other  analysts  and  to 
estimate  users.  It  Is  likely  to  have  very  little  Impact,  except  on  Its  origina¬ 
tor.  Furthermore,  It  Is  I  I  ke  I  y  that  the  I  ns  I  ghts  of  one  analyst  are  s  I  gn  I  f  I  — 
cant  I y  sharpened  and  refined  when  they  are  shared  with  and  examined  by  others. 
A  theory  developed  I  n  a  commun  I  ty  of  ana  I  ysts  Is  I  Ikel  y  to  be  better  than  the 
same  theory  developed  only  by  Its  originator. 

This  suggests  that  there  are  Important  social  and  organizational  dimensions 
to- theory  formulation,  revision,  and  change;  and  that  aiding  these  dimensions  of 
estimation  goes  beyond  the  Individual  analyst  (see  Stech,  1979,  for  an  appraisal 
of  social,  organizational,  and  political  aspects  of  Intention  estimation).  Just 
as  there  are  weaknesses  In  estimation  logic,  there  are  weaknesses  In  the  organi¬ 
zational  processes  of  estimation.  These  social  and  organizational  dimensions 
become  Important  when  the  Individual  analyst  Is  developing  a  new  theoretical 
outlook  that  conflicts  with  or  goes  beyond  the  normal  accepted  theory,  l.e,, 
when  the  analyst’s  new  theory  conflicts  with  other  analysts’  theories. 

Most  of  the  advice  given  In  this  chapter  Involves  greater  precision  of 
hypotheses,  greater  specification  of  variables  and  relationships,  more  use  of 
mathematical  and  psychological  techniques  to  ascertain  and  refine  variables  and 
relationships,  and  greater  willingness  to  predict  and  check  predictions.  All  of 
this  advice  amounts  to  saying  that  naval  analysts  might  do  Intention  estimation 


more  like  scientists  do  science.  Quantification,  measurement,  specificity  and 
precision,  and  prediction  are  means  and  ends  of  the  methods  and  tools  of 
science,  he  or  she  can  f I nd  a  I arge  I Iterature  of  helpful  and  useful  advice  on 
theory  building  and  testing,  scientific  method,  epistemology  and  the  philosophy 
of  science  and  scientific  theory.  There  Is  no  point  attempting  to  review  the 
pathways  Into  that  literature  here,  any  reader  who  avails  himself  of  the 
references  In  this  study  will  quickly  find  many  leads  If  such  are  needed. 

A  more  Important  Issue  Is  whether  and  to  what  degree  scientific  methods  are 
appropriate  for  the  social,  political,  and  naval  science  questions  that  concern 
the  naval  analyst.  Science  requires,  In  addition  to  the  attributes  noted  above, 
control  and  experimental  manipulation  of  variables.  Control  and  experimentation 
are  rarely  possible  In  the  social  sciences  to  the  degree  that  they  can  be 
exercised  In  physical  sciences.  Qoes  this  mean  that  the  social  sciences  cannot 
be  truly  scientific  and  that  attempts  at  scientific  methods  In  these  areas  are 
fated  to  be  little  more  than  over-sophistication? 

No  clear-cut,  general  prescrl ptions  can  be  offered  to  the  analyst  on  this 
Issue.  It  will  be  up  to  the  analyst  whether  to  adopt  a  scientific  or  quantita¬ 
tive  technique,  to  attempt  explicit  theorizing  and  theory  building,  or  to  con¬ 
tinue  to  rely  on  narrative  explanation.  Scientists  and  philosophers  continue  to 
wrestle  with  the  question  of  whether  social  science  can  be  truly  scientific 
(e.g.,  see  McCIIntock,  1981}  Zlman,  1979).  Just  as  unaided  analysis  and  theory- 
building  have  their  limitations,  as  outlined  In  Stech  (1981,  Chapter  4),  scien¬ 
tific  methods  and  theories  have  their  l!m;ts,  especially  as  applied  to  social 
phenomena.  Naval  analysts  should  be  awarr  of  the  limits  of  both. 

Whether  a  particular  quantitative  aid  or  theory-building  approach  will  be 
useful  or  helpful  Is  largely  an  empirical  question.  We  have  tried  to  outl  Ine 
how  the  naval  analyst  can  detect  and  assess  the  shortcomings  of  his  or  her 
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unaided  est I  mat  Ion  or  narrative  logic.  The  analyst  will  have  to  apply  this 
general  knowledge  to  the  particular  case  to  determine  whether  the  shortcomings 
thus  detected  are  serious  enough  to  warrant  the  use  of  expl  tclt  quantitative 
aids.  Similarly,  the  anal,jt  will  have  to  evaluate  the  advantages  against  the 
costs  of  developing  a  quantitative  theory-building  approach,  the  use  of  multiple 
hypothesis  dlsconf Irmatlon  strategies  or  other  theory-revision  methods  noted  In 
the  previous  sections  of  this  chapter. 

In  effect,  the  analyst  facss  an  Infinite  regress:  how  to  decide  whether  or 
not  to  try  out  a  particular  method  or  theory-bu! I  ding  approach?  If  decision- 
aiding  methods  or  judgment-enhancing  approaches  are  recommended  to  help  with 
this  first  question,  the  analyst  Is  faced  with  the  second  question:  how  to 
decide  whether  an  aided  decision  Is  better  than  an  unaided  one? 

While  it  Is  a  basic  finding  of  this  study  that  unaided  estimation  tends  to 
be  Inferior  to  estimates  that  explicitly  employ  aids  to  Information  organiza¬ 
tion,  Integration,  and  Inference,  In  general  we  cannot  draw  hard  conclusions 
about  how  Inferior  unaided  estimation  might  be.  This  makes  the  question  of  the 
costs  and  henef Its  of  aided  versus  unaided  estimation  and  theory-bu 1 1 d I ng  an 
empirical  one  and  one  for  which  the  analyst  might  want  to  seek  out  decision- 
aiding  or  Judgment-a! ding  expert l se,  parti cu I ar ly  If  the  costs  and  risks  In¬ 
volved  are  very  high. 

The  approach  taken  by  O'Leary  and  Cop  1 1 n  (1975)  seems  an  estimable  one 
worth  repeating  (although  perhaps  with  a  more  systematic  appraisal  methodology). 
They  compared  the  qualitative  analyses  and  forecasts  of  unaided  State  Department 
Intelligence  analysts  to  estimations  based  on  quantitative  approaches  to  the 
same  Intel  I  Igence  Issues.  They  also  evaluated  the  costs  and  benefits  of  the 
quantitative  methods.  O'Leary  and  Copltn  made  an  Informal  cost-benefit 
analysis,  and  It  might  be  necessary  for  naval  analysts  to  make  such  evaluations 
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more  explicitly  and  systematically  (e.g.,  using  cost-oeneflt  techniques, 
decision-analysts,  mul tlattrlbute  utility  methods,  etc,)  or  to  attempt  a 
prospective  rather  than  retrospective  evaluation.  The  main  point,  however,  Is 
that  the  quantitative  methods  were  tried  to  see  If  they  work,  with  what  limits, 
to  determine  how  much  Improvement  they  could  make,  and  at  what  expense.  This  Is 
an  approach  we  would  recommend  to  any  analyst  who  mtght  be  tempted  to  try  an 
Improved  methodology  for  Intention  estimation. 
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IV.  CATALOGUE  OF  AIDS  AND  PROCESS  SUMMARY 


The  analytical  aids  discussed  and  evaluated  In  the  previous  sections  of  the 
report  are  catalogued  for  convenient  reference  In  this  section.  The  Items 
covered  In  the  one-page  summaries  correspond  closely  to  the  criteria  outlined  In 
Chapter  III.  Pr  ob I  ems,  pitfalls,  and  warn  t  ngs  re  I ated  to  specific  aids  are 
summarized  In  the  "Comments"  section  of  each  summary. 

No  study  of  this  nature  and  scope  could  produce  an  exhaustive  survey  of  all 
possible  analytical  aids.  Further,  the  state-of-the-art  of  Information  handling 
and  Interpretation  is  changing  quite  rapid./.  To  accomodate  future  additions, 
updates,  and  enhancements  to  the  catalogue,  the  one-page  summary  format  has  been 
emp l oyed . 

The  catalogue  of  aids  Is  organized  Into  four  basic  sections  Including: 

<  A.  Judgemental  methods  of  both  an  explicit  and  Implicit  nature. 

B,  Specialized  analytical  aids  to  support  specific  elements  of  the 
estimation  process, 

C,  Extrapolation  methods  and  models  that  rely  primarily  on  past 
events  and  experiences  as  an  Information  base. 

D,  Structural  reoresentat I ons  and  models  which  emphasize  causal 
relationships  and  physical  constraints. 


A.  JUDGEMENTAL  METHODS 


Methods  catalogued  In  this  section  assist  In  the  harnessing  of  Individual 
and  group  Judgement  to  deal  with  estimation  problems.  Emphasis  Is  placed  In 
these  methods  on  the  organization  of  Individuals  and  groups  to  take  advantage  of 
overlapping  capabilities,  to  fill  In  gaps  In  capabilities  and  experience,  and  to 
systematize  the  judgemental  process. 

Behavioral  science  studies  have  revealed  several  Impl left  patterns  that 
arise  In  Judgemental  processes,  while  these  often  appear  more  In  the  nature  of 
mental  traps  than  aids,  their  Identification  In  this  catalogue  can  help  to 
maintain  an  awareness  of  potential  problems  that  may  arise  and  to  use  these' 
Implicit  patterns  In  a  more  constructive  way. 
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EXPERT  OPINION 


Category ;  Perceiving  data,  prediction. 

Scope; 

Method  can  cover  any  aspect  of  problem  where  expertise  has  been  developed. 


Based  on  human  perceptions,  Interpretations,  and  reliability.  Most  prob¬ 
lems  will  Involve  group  of  experts  with  some  overlapping  capab I  I  1 1 1 es  and, 
probably,  some  gaps  In  coverage. 


Comanta: 

Method  suffers  from  difficulties  In  assembling  and  Integrating  opinions 
from  group  members  dealing  with  complex  problems.  Severe  difficulties  In  deter- 
mlng  logical  process  and  biases  that  affect  opinion  of  Individual  or  group. 


I 
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ANALYTICAL  AID:  DELPHI  TECHNIQUE 


Category ;  Characterizing  data,  prediction. 


Scooe: 


Represents  structural  approach  to  use  of  Individual  and  group  expert  opin¬ 
ion  with  provision  for  feedback  and  learning. 

Logic  Structure,  and  Methodology: 

Responses  to  well  defined  questions  or  problems  are  Integrated  and  returned 
to  contributors  with  full  Information  on  the  comparison  of  their  es+Imates  with 
others  In  group.  Responses  can  be  changed  In  proceeding  rounds. 


Add! I  cat Ions: 

Classifying  Soviet  Navy  Into  component  missions  (Thorpe,  NPSP,  ch.  8). 


Document  at  I  on .  V  al  I  dat  1  on_._  Rev  Law : 
Morgenstern,  Knorr,  Helss,  1973:  23-26 


CflananTa: 

Range  of  expert  opinion  generally  converges  sharpiy,  thereby  possibly 
masking  real  uncertainty.  Method  Is  costly  and  time  consuming;  reasons  for 
shifts  In  responses  are  not  usually  apparent. 


113 


SCENARIO  METHODS 


Category :  Characterizing  data,  prediction. 


Scone; 

Involves  complete  description  of  outcomes  that  may  be  anticipated  with 
explicit  representation  of  ail  causal  and  contributing  variables  In  a  consistent 
framework. 


Scenario  description  usually  Includes  quantitative  and  qualitative  Informa¬ 
tion.  Explicit  nature  of  description  facilitates  third-party  review  and  com¬ 
ment. 


None  Cited. 


Comments: 

Useful  method  to  test  consistency  and  plausibility  of  analysis.  Complemen¬ 
tary  to  expert  opinion,  Delphic,  and  most  modeling  efforts. 


ANALYTICAL  AID;  AVAILABILITY  AND  REPRESENTATIVENESS  HEURISTIC 


Category :  Precelvlng  data 


Scope; 

This  heuristic  Involves  Intuitive  attributes  which  exist  In  most  perception 
and  Interpretation  problems.  WhI I e  not  real ly  analytical  methods,  they  are 
Implicit  In  analysis  and  must  be  Included  here. 


Logic  Structure,  and.  Methodology; 

The  representative  heuristic  Is  the  tendency  to  assume  that  a  sample  Is 
representative  of  the  populat'on  from  which  It  is  drawn,  and  to  neglect  features 
of  the  population  that  are  not  In  the  sample.  The  availability  heuristic  deals 
with  the  tendency  to  concentrate  on  information  that  Is  accessable  and  to  limit 
the  scope  of  search  for  complementary  or  supplementary  information. 


Acal  Lc.atians: 

Most  judgemental  estimates. 


Comments : 

This  Implicit  behavior  Is  a  factor  In  most  Judgemental  estimates  and  must 
be  used  constructively. 
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B.  SPECIALIZED  ANALYTICAL  AIDS 


This  group  of  analytical  aids  deals  with  specific  portions  of  the  estima¬ 
tion  process  and  emphasize  the  perception,  characterization,  and  organization  of 
Information  rather  than  prediction. 

Some  of  the  methods  described  in  this  section  can  be  used  to  support  and 
supplement  the  Judgemental  methods  of  the  previous  section,  while  others  are 
Intended  to  be  used  separately  from  judgemental  methods. 


ANALYTICAL  AID:  CODING  OF  EVENTS  (EVENT  CODING) 


Category :  Perceiving  Data,  Characterizing  Data 

Scope; 

Method  can  be  applied  to  any  stream  of  Information  involving  series  of 
discrete  action  events. 


Logic  Structure  and_.Methodolodv; 

Coded  events  are  amenable  to  psychological  analysis  and  I nterpretat I  on. 
Different  coding  categories  may  be  used  for  expected  and  unexpected  events. 
Method  can  capture  gross  relationships  between  events  and  finer-grained  details 
of  actions.  Interruptions,  ommissians,  and  non-occurrences  may  be  detected 
agains  an  orderly  background  of  actions.  Requires  varying  levei  of  tnput  data 
and  Is  totally  dependent  on  that  data. 


Apal.tea.tl.Qiis: 

Conf I  let  and  sea  I e  of  viol ence  (Egypt- Israel)  (O'Leary  and  Cop  I i n,  1975). 
Crisis  forecasting  and  prediction  (March  1977  Issue  of  International  Studies 
Quarterly).  Soviet  naval  operations  (Dismukes  and  McConnell,  1979;  ch.  2  by 
Petersen ) . 


Comments: 

Method  Is  Important  In  the  organization  of  Information  based  on  observa¬ 
tions  in  most  estimation  applications.  Analyst  coding  events  may  be  affected  by 
a  number  of  perception  phenomena. 
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CONTENT  ANALYSIS  (FACTOR  ANALYSIS) 


1C-  V 


Category :  Precelving  Oata,  Characterizing  Oata 


Scope; 

Generally  applied  to  verbal  or  written  statements,  e,g.,  propaganda,  spe¬ 
eches,  memoirs. 


Method  uses  statistical  techniques  to  determine  objective  frequencies  for 
actions,  events,  or  statements  to  determine  such  things  as  authenticity,  trends 
In  semantics  or  rhetoric,  and/or  shifts  In  Interests.  Method  Is  systematic  and 
Is  based  on  clear  rules  of  coding  and  Inference, 


Objective  estimation  of  frequencies  for  actions,  events,  and/statements 
(George,  1959).  Soviet  Position  In  Law  of  the  Sea  negotiations  (Frledhelm  and 
Jehn,  1977).  Attitudes  of  Soviet  elites  (Heuer,  1978).  Nazi  Intentions 
(George,  1959).  Soviet  political  elite  speeches  (Kirk,  1980).  Warship  design 
(Kehoe,  SNI,  ch.  19). 


Comments; 

Method  Is  very  labor  Intensive  and  time  consuming.  Content  can  sometimes 
be  quite  subjective  requiring  careful  training  and  monitoring  of  analyst. 
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ANALYTICAL  AID:  PARAMORPHiC  REPRESENTATIONS 


Category :  Perceiving  Data 


Logic  Structure  and  Methodology: 

Method  Is  Idealized  representation  of  group  of  experts  assumtng  that  a 
model  capturing  the  essential  elements  of  their  background  Is  more  consistent 
than  human  Integration.  Intended  to  overcome  human  distortions  and  Inconsis¬ 
tencies  attributable  to  boredom,  fatigue,  distractions,  and  variable  applica¬ 
tions  of  Integration  rules.  Mathematics  usually  Involve  simple  linear  combina¬ 
tions  of  predictive  variables. 


Ado  Meat  ions: 
None  noted. 


Documentation: 

Hoffman,  1960.  SIovIc,  1969. 


Comments : 

Idealized  model  does  not  capture  insights  and  Innovative  findings  that  may 
occur  In  human  expert  process.  Not  felt  to  be  appropriate  for  complex,  non¬ 
linear,  configural  tasks. 


ANALYTICAL  A  IQ:  BOOTSTRAPPING 


Category ;  Perceiving  data.  Assessing  Covariation,  prediction. 


Scope: 


Extension  of  paramorphic  representation  to  use  Judgemental  data  as  input  to 
model  also  incorporating  real  data  and  observations. 

Logic  Structure  and  Methodology: 

Linear  models  are  fitted  to  past  judgements.  Idea  of  modeling  the  judge's 
Judgements  and  then  using  the  Judgement  of  the  model  leads  to  term  "bootstrap¬ 
ping". 


Add  I  1  cations: 

Instability  In  tropfcai  Africa  (Cleary  and  Coplin,  1975). 


Documentation: 

Dawes,  1971.  Goldberg,  1970. 


Comments j 

Provides  systematic  data-driven  means  to  appraise  suspected  covariations 
that  resolve  some  problems  with  theory-driven  appraisal.  Reliability  of  model 
must  be  considered  In  any  application. 
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ANALYTICAL  AID;  METRIC  AND  NON-METRIC  MULTIDIMENSIONAL  SCALING, 

TREE  FITTING,  ANO  CLUSTERING 


Category ;  Perceiving  Data.,  Characterizing  Data. 


Scope: 

Developed  to  deal  with  system  I zat I  on  of  expert  opinions  on  the  features  of 
events  or  other  samples. 

Logic  Structure  and  Methodology; 

Method  Is  aid  In  constructing  meaningful  perceptual  categories  for  analyst. 
Uses  computer  technique  for  representation  of  psychological  or  physical  struc¬ 
ture. 


Applications: 

Assess  similarity  between  pairs  of  stimuli  and  sorting  of  classlf Icav!on 
stimuli  Into  categories  of  Soviet  general  purpose  platforms  (Tnorpe).  Soviet 
naval  behavior  (Dtsmukes  and  McConnel).  Identification  of  voting  blocks  In  U.N. 
(Heuer,  1978,  ch.  8). 


Comments: 

Appears  to  be  more  promising  than  Delpht  because  method  attempts  to  make 
structure  of  problem  and  assumptions  more  explicit.  See  Shepard  (1980). 
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C.  EXTRAPOLATION  ME7TH00S  ANO  MOOELS 

The  methods  described  In  this  section  use  statistical  and  regression 
techniques  to  extrapolate  future  patterns  and  behavior  based  on  past 
observations  and  experience.  These  methods  are  general  ly  used  on  aggregated 
data. 

The  principal  objective  of  these  methods  Is  to  assist  In  the  Identification 
of  cause-effect  relationships,  covariance  analysis,  and  prediction.  The  methods 
are  most  app 1 1 cab  I e  where  It  Is  difficult  or  Impossible  to  arrive  at  an  expl Ici+ 
representation  of  the  underlying  structure  affecting  a  problem. 

Both  determ  I n I st I c  and  probabllstlc  techniques  may  be  employed  with  these 
methods. 
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LINEAR  REGRESSION 


<i 


Category ;  Weighting  data.  Cause-effect  relationship. 


Scope; 

Method  can  test  and/or  establish  relationships  between  sets  of  quantitative 
variables. 


I  I  1 1  rK!liiln«ulK7n 


Uses  well  developed  statistical  techniques  to  establish  relationships  and 
Indicate  error  range  of  estimation.  Related  to,  but  more  comprehensive  than 
time- trend  method. 


Variety  of  quantitative  estimation  problems. 


ANALYTICAL  AID:  TIME  TREND  LINE 


Category ;  Assessing  Cover  I  at  lore 


Scope i 

Oesc"  «*  trend  over  time  for  any  defined  parameter  or  measurable  event. 
Logic  Structure  and  Methodology. 

Graphical  representation  of  parameter  or  measurable  event  over  time. 


Aool (cations: 

Capabl  I  1  ty  of  U.S.  and  Sov  let  ship  types  (Kehoe,  SN  I ,  ch.  8).  latt  n  Amei — 
lea  (O'Leary  and  Coplln,  1973,  ch.  8)  mlllatry  expenditures  and  other  variables. 


Coamtits: 

Useful  display  format.  May  lead  to  regression  model  based  on  actual  rela¬ 
tionship.  Useful  for  Identifying  shifts  In  pattern  of  events. 


ANALYTICAL  AID:  CAUSAL  ANALYSIS 


Category  t  Cause-effect  Assessment 


Scope : 


Encompasses  systematic  assessment  of  causal  relationships  In  nonexperl- 
mental  situations. 


Lflfllc.  Structure, 

Method  sel ects  var I abl es  that  are  potentl al  determ  I nants  of  effects  and 
Isolates  separate  contributions  to  effects  made  by  each  suspected  cause.  Typi¬ 
cally  use  regression  analysis. 


Anpl I cat  Ions: 

Coal  1 1 1 ons  among  ol 1 -export  I ng  and  o  1 1  - 1  m port  I ng  count r I es  (O'Leary  and 
Copl  I  n,  1  975,  ch.  7). 

QoaifflflfltatlQn.  Validation.  Review: 

Asher,  1976.  Blalock,  1964.  Helse,  1975. 

Comments: 

Similar  to  regression  analysis  and  very  data  dependent. 
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ANALYTIC  AID;  ACTUARIAL  M0GEL3 


Category ;  Assessing  Covariations 


Scope : 

Can  be  applied  to  any  estimation  problem  Involving  relationship  of  outcome 
variables  to  measurable  or  observable  variables. 


Loal.C-Structura.  anrl  Methodo  I  ogv ; 

Uses  regression  analysis  to  relate  response  or  outcome  variables  to  mea¬ 
surable  variables  to  determine  combinational  rule. 


Applications: 
None  noted. 


Cannsnls: 

Provides  Improvement  over  Judgemental  approaches  to  synthesizing  Input  data 
to  estimate  outcome. 
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ANALYTICAL  AID;  BACK CASTING 


Category :  Prediction 


i 

Validation  method  for  model  or  mathematical  aid  based  on  regression,  econo¬ 
metric,  or  structural  model. 


Logic  Structure  and  Methodology: 

Method  determines  whether  assumptions  underlying  the  forecast  are  true  for 
past  and  present  data.  Forecasters  often  fall  to  adjust  their  forecasts  In  a 
direction  that  would  rectify  earl  !er  forecast's. 


Applications: 

Several  failures  to  use  this  technique,  leading  to  underestimation  of 
Soviet  missile  Inventories,  are  noted  (Sullivan,  In  Godson,  1980;  Wehlstetter, 
1975a). 


Documentation: 

Asher,  1978.  Mcrgenstern,  Knorr,  and  Helss,  1973. 


Comments: 

Important  to  use  this  as  method  of  validating  and  correcting  any  model. 
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STRUCTURAL  REPRESENTATIONS  ANO  MODELS 


This  class  of  analytical  methods  probes  more  deeply  Into  causal  and  struc¬ 
tural  relationships  of  a  phys'cal  and  conceptual  nature.  The  motivation  for 
this  Is  to  develop  a  more  representatl ve  framework  within  which  the  effect  of  a 
number  of  variables  and  uncertain  assumptions  may  be  tested. 

These  methods  are  most  appropriate  where  significant  patterns  or  paradigm 
changes  mights  be  anticipated.  Since  physical  relationships  and  constraints 
(force  structure,  weapons  capabilities,  logistic  capabilities)  are  generally 
easy  to  capture  In  this  type  of  f ramework,  the  methods  tend  to  be  applied  in 
cases  where  pnyslcal  capabilities  have  a  strong  Influence,  or  are  a  constraining 
factor,  on  Intentions  and  actions. 

Analytical  methods  In  this  category  may  be  deterministic  or  probabl Istlc. 
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ANALYTICAL  AID:  STRUCTURAL  OR  EXPLICIT  MODELING  (THEORIES) 


Category :  Cause-effect  Assessment.  Prediction 


Soogg; 

Method  attempts  to  represent  real  structural  relationships  involving 
behaviors)  and  physical  elements  pertinent  to  estimation  problem. 


L flSiC.  Structure  and.  Methodology; 

Model  representation  may  be  theoretical  (relationships  are  postulated  and 
then  studied  by  applying  model)  or  based  on  understanding  of  actual  structure. 
This  class  of  modeling  may  be  normative  in  form  (indicates  what  Intentions 
should  be  given  constraints  and  physical  capabilities)  or  simulative  (what  might 
be  expected  given  certain  assumptions).  The  normative  form  uses  optimization 
techn I ques,  while  the  simulation  methods  can  involve  a  number  of  analytical 
methods  including  econometrics  and  system  dynamics.  Methods  make  extensive  use 
of  mathematical  techniques.  Feedback  relationships  may  be  modeled  explicitly 
using  system  dynamics  methods  or  other  methods  derived  from  control  theory. 


Add  1 1 cat ions: 

Arms  race  models  (Wallace,  1979).  Instability  in  Tropical  Africa  (O'Leary 
and  Cop  I  In,  1975). 


Comments- 

Structural  modeling  Is  most  applicable  In  dealing  with  combined  physical 
and  behavioral  factors.  It  has  strong  promise  in  being  able  to  deal  directly 
with  situations  where  physical  capabilities  (force  structure,  logistics,  weapons 
capabl I  tty;  either  may  reveal  Intentions  or  place  significant  constraints  on 
Intentions. 
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ANALYTICAL  AID:  PARAOIGM  SHIFTS 


Category :  Prediction 


&£££: 

Any  relationship,  no  matter  how  wel  I  It  appears  to  be  establ  Ished  may  be 
subject  to  a  paradigm  shift  and  this  fact  must  be  considered  tn  any  changing 
environment. 


Logic  Structure  and  Methodology; 

Paradigm  shifts  can  affect  the  validity  of  any  model  or  analytical  method 
that  does  not  explicitly  take  the  possibility  Into  account. 

Aopl 1  cat  Ions? 

None  noted. 


Documentation ...  Val  I  dat  I  on .  Rev  I  ew : 

Kuhn,  1970. 

Comments: 

No  reliable  methods  have  been  developed  +o  assist  In  the  Identification  of 
such  shifts;  they  may  be  dealt  with  In  a  fashion  by  expert  opinion  and  time- 
trend  methods. 
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ANALYTICAL  AID;  SEARCH  TREES  (FAULT  TREE  ANO  DECISION  TREE) 


Cause-effect  Assvessment 


Scope: 

Oescrtbes  any  process  with  sequentially  related  elements. 


Logic  Structure  and  Methodology; 

Sequentially  related  elements  are  represented  explicitly  In  tree  structure 
concerning  dual,  outcomes  or  uncertain  outcomes  and  consequences.  Well  developed 
mathematical  techniques  are  available  for  probabl  I  istic  analysis,  search  and 
optimization  routines. 


Applications: 

None  noted. 

Documentation: 

Posner,  1973:  154-158 

Comments: 

Powerful  technique  for  organizing  relevant  Information  on  complex 
processes.  May  be  used  with  Bayesian  analysis  and  other  probabal Istfc  methods. 
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ANALYTICAL  AID :  BAYESIAN  ANAL YS I S 


Category ;  Weighting  Oata 


Scope; 


Oeals  with  any  probabal I stl  c  problem  where  conditional  estimates  may  be 
developed. 

Logic  Structure  and  Methodology; 

Considers  probabal I stlc  relationships  between  event  and  causal  or 
contributing  parameters.  Problems  structured  In  this  way  can  be  dealt  with  by 
extensive  and  powerful  mathematical  tools. 


AddI I cat Ions; 

Estimation  of  probab I i 1 1 1 ons  of  MI d-East  conf I  I ct  (Heuer,  1973,  ch.  2  by 
Schweitzer).  Analysis  of  order  of  battle  data  (North  VIetman,  USSR-Chlna,  Arab 
Israel  conflict).  Integration  of  judgemental  information  (Morris,  1974,  1976). 


Documentation.  Validation.  Rav L$s. ; 

C I ov I c  and  Lichtenstein,  1971.  Edwards,  1978. 


Comments : 

Method  helps  fo  Integrate  probabal I stlc  data  Into  Judgements.  Also  useful 
In  combining  estimates  of  different  experts  and  can  be  used  with  decision  tree 
methods.  Can  estimate  subjective  probability  that  a  causal  explanation  for  an 
event  Is  true. 
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